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5 MACHINE VISION CALIBRATION TARGETS AND METHODS 

OF DETERMINING THEIR LOCATION AND ORIENTATION IN AN IMAGE 

Reservation of Copyright 

10 The disclosure of this patent document contains material which is subject to 

copyright protection. The owner thereof has no objection to facsimile reproduction by 
anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and 
Trademark Office patent file or records, but otherwise reserves all copyright rights 
whatsoever. 

15 

Background of the Invention 

The invention pertains to machine vision and, more particularly, to calibration 
targets and methods for determining their location and orientation in an image. 

20 

Machine vision refers to the automated analysis of an image to determine 
characteristics of objects and other features shown in the image. It is often employed in 
automated manufacturing lines, where images of components are analyzed to determine 
placement and alignment prior to assembly. Machine vision is also used for quality 
25 assurance. For example, in the pharmaceutical and food packing industries, images of 

packages are analyzed to insure that product labels, lot numbers, "freshness" dates, and the 
like, are properly positioned and legible. 

In many machine vision applications, it is essential that an object whose image is 
30 to be analyzed include a . calibration target. Often a cross-shaped symbol, the target 
facilitates determining the orientation and position of the object with respect to other 
features in the image. It also facilitates correlating coordinate positions in the image with 
those in the "real world," e.g., coordinate positions of a motion stage or conveyor belt on 
which the object is placed. A calibration target can also be used to facilitate determining 
35 the position and orientation of the camera with respect to the real world, as well as to 
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facilitate determining the camera and lens parameters such as pixel size and lens 
distortion. 



In addition to cross-shaped marks, the prior art suggests the use of arrays of dots, 
5 bulls-eyes of concentric circles, and parallel stripes as calibration targets. Many of these 
targets have characteristics that make difficult finding their centers and orientations. This 
typically results from lack of clarity when the targets and, particularly, their borders are 
imaged. It also results from discrepancies in conventional machine vision techniques used 
to analyze such images. For example, the edges of a cross-shaped target may be 
10 imprecisely defined in an image, leading a machine vision analysis system to wrongly 

interpret the location of those edges and, hence, to misjudge the mark's center by a fraction 
of a pixel or more. By way of further example, a localized defect in a camera lens may 
cause a circular calibration mark to appear as an oval, thereby, causing the system to 
misjudge the image's true aspect ratio. 

15 

In addition to the foregoing, many of the prior art calibration targets are useful 
only at a limited range of magnifications. Parallel stripes, for example, do not provide 
sufficient calibration information unless many of them appear in an image. To 
accommodate this, a machine vision system must utilize lower magnification. However, 
20 as the magnification decreases, so does the ability of the machine vision equipment to 

distinguish between individual stripes. Similar drawbacks limit the usefulness of the other 
prior art calibration targets for use in all but a narrow range of magnifications. 

Though the art suggests the use of checkerboard patterns as alignment marks, the 
25 manner in which images of those marks are analyzed by conventional machine systems 
also limits their utility to a limited range of magnifications. Particularly, prior art systems 
obtain alignment information from checkerboard marks by identifying and checking their 
corners, e.g., the eight black (or white) corners in a black-and-white image. By relying on 
corners, the systems necessitate that images show entire checkerboards, yet, with 
30 sufficient resolution to insure accurate detection and analysis. 
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An object of this invention is to provide an improved calibration targets ana 
methods for machine vision analysis thereof. 



A related object is to provide calibration targets and analysis methods reliable at a 
5 wide range of magnifications. 

A further object is to provide such methods as can be readily implemented on 
conventional digital data processors or other conventional machine vision analysis 
equipment. 

10 Yet still another object of the invention is to provide such methods that can rapidly 

analyze images of calibration target without undue consumption of resources. 
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Summary of the Invention 

The foregoing objects are among those attained by the invention, which provides 
in one aspect a machine vision method for analysis of a calibration target of the type 
having two or more regions, each having a different "imageable characteristic" (e.g., a 
different color, contrast, or brightness) from its neighboring region(s). Each region has at 
least two edges — referred to as "adjoining edges" — that are linear and that are directed 
toward and, optionally meet at, a reference point (e.g., the center of the target or some 
other location of interest). The method includes generating an image of the target, 
identifying in the image features corresponding to the adjoining edges, and determining 
the orientation and/or position of the target from those edges. 

In another aspect, the invention provides a method as described above for 
analyzing a target of the type that includes four regions, where the adjoining edges of 
each region are perpendicular to one another, and in which each region in the target has a 
different imageable characteristic from its edge- wise neighbor. The edges of those regions 
can meet, for example, at the center of the target, as in the case of a four-square 
checkerboard. 

In yet another aspect, the invention provides a method as described above for 
determining an orientation of the target as a function of the angle of the edges identified in 
the image and for determining the location of the reference point as an intersection of lines 
fitted to those edges. In regard to the former, the invention provides a method of 
determining the orientation of a target in an image by applying a Sobel edge tool to the 
image to generate a Sobel angle image, and by generating a angle histogram from that 
angle image. In an alternate embodiment, the orientation is determined by applying a 
Hough line tool to the image and determining the predominant angle of the edges 
identified by that tool. 

In regard to the location of the reference point, one aspect of the invention calls for 
locating the adjoining edges by applying a caliper vision tool to the image, beginning at an 
approximate location of the reference point. That approximate location of the reference 
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point can itself be determined by applying a Hough line vision tool to the image in order 
to find lines approximating the adjoining edges and by determining an intersection of 
those lines. Alternatively, the approximate location of the reference point can be 
determined by performing a binary or grey scale correlation to find where a template 
representing the edges most closely matches the image. 



In another alternate embodiment, the approximate location of the reference point is 
determined by applying a projection vision tool to the image along each of the axes with 
which the adjoining edges align. A first difference operator vision tool and a peak 
10 detector vision tool are applied to the output of the projection tool (i.e., to the projection) 
in order to find the approximate location of the edges. 

The invention has wide application in industry and research applications. It 
facilitates the calibration of images by permitting accurate determination of target location 
15 and orientation, regardless of magnification. Thus, for example, an object bearing a target 
can be imaged by multiple cameras during the assembly process, with accurate 
determinations of location and orientation made from each such image. 

These and other aspects of the invention are evident in the drawings and in the 
20 description that follows. 
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Brier Description of the Drawings 

A more complete understanding of the invention may be attained by reference to 
the drawings, in which: 

5 

Figure 1A - 1C depict calibration targets according to the invention; 

Figure ID depicts the effect of rotation on the target depicted Figure IB; 

10 Figure 2 depicts an object according to the invention incorporating a calibration 

target of the type depicted in Figure IB; 

Figure 3 depicts a machine vision system according to the invention for 
determining the reference point and orientation of a calibration target; 

15 

Figures 4 and 5 depict a method according to the invention for interpreting an 
image of a calibration target to determine a reference point and orientation thereof; and 

Figure 6 illustrates the magnification invariance of a target according to the 
20 invention. 
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Detailed Description of the Dlustrated Embodiment 

Figures 1 A - 1C depict calibration targets according to the invention. Referring to 
Figure 1 A, there is shown a target 10 according to the invention having three regions 12, 
5 14, 16. Each region is bounded by at least two linear edges that are oriented toward a 
reference location or reference point 18 on the target. Thus, for example, region 12 is 
bounded by edges 20, 24; region 14 is bounded by edges 20, 22; and region 16 is bounded 
by edges 22, 24. As evident in the drawings, the edges are shared by adjoining regions 
and, hence, are referred to below as "adjoining edges." Thus, region 12 shares edge 20 
10 with region 14; region 14 shares edge 22 with region 16; and region 16 shares edge 24 
with region 12. In the illustration, the reference point 14 is at the center of target 10, 
though, those skilled in the art will appreciate that the reference point can be positioned 
elsewhere. 

15 Each of the regions has a different imageable characteristic from its neighboring 

regions. As used herein, an "imageable characteristic" is a characteristic of a region as 
imaged by a machine vision system (e.g., of the type shown in Figure 3) and, particularly, 
as imaged by an image capture device used by such a system. For example, in the 
illustration, region 12 has the characteristic of being colored black; region 14, white; and 

20 region 16, gray. In addition to color, imageable characteristics useful with conventional 
machine vision systems — which typically utilize image capture devices operational in 
visual spectrum — include contrast, brightness, and stippling. 

Those skilled in the art will appreciate that any other characteristics by which a 
25 region may be identified and distinguished in an image are suitable for practice of the 
invention. Thus, for example, for a machine vision system that utilizes a temperature- 
sensitive (or infrared) image capture device, an imageable characteristic is temperature. 
By way of further example, for a machine vision system that utilizes a nuclear decay 
radiation-sensitive image capture device, an imageable characteristic is emitted radiation 
30 intensity or frequency. 
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As shown in the illustration, the adjoining edges 20, 22, 24 comprise straight linear 
segments. Those edges are implicitly defined as the borders between regions that, 
themselves, have different imageable characteristics. Thus, for example, edge 20 is a 
straight linear segment defined by the border between black region 12 and white region 
5 14. Likewise, edge 24 is defined by the border between black region 12 and gray region 
16. Further, edge 22 is defined by the border between white region 14 and grey region 16. 

Figure IB depicts a calibration target 30 according to the invention having four 
rectangular (and, more particularly, square) regions 32, 34, 36, 38. As above, each region 

10 is bounded by at least two linear edges that are oriented toward a reference point 40 at the 
center of the target. Thus, for example, region 32 is bounded by edges 42, 44; region 34 is 
bounded by edges 42, 46; and so forth. As above, these edges are shared by adjoining 
regions. Thus, region 32 shares edge 42 with region 34, and so forth. Each region in 
target 30 has a different imageable characteristic from its edge-wise neighbor. Hence, 

15 regions 32 and 36 are white, while their edge-wise adjoining neighbors 34, 38 are black. 

Figure 1C depicts a calibration target 50 according to the invention having five 
regions 52, 54, 56, 58, 60, each having two linear edges directed toward a reference point 
62. The adjoining regions are of differing contrast, thereby, defining edges at their 
20 common borders, as illustrated. Although the edges separating the regions 52 - 60 of 

target 50 are directed toward the reference point 62, they do not meet at that location. As 
evident in Figure 1C, no marker or other element imageable characteristic is provided at 
reference point 62. 



25 Those skilled in the art will appreciate that, in addition to the calibration targets 

shown in Figures 1 A - 1C, targets with still more regions (or as few as two regions) and 
shapes, otherwise in accord with the teachings hereof, fall within the scope of the 
invention. Moreover, it will be appreciated that targets may be of any size and that their 
regions need not be of uniform size. Still further, it will be appreciated that the outer 

30 borders of the targets need not be linear and may, indeed, take on any shape. 
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Figure 2 depicts an object according to the invention for use m machine vision 
imaging, detection, and/or manipulation having a calibration target according to the 
invention coupled thereto. In the illustration, the object is an integrated circuit chip 70 
having coupled to the casing thereof a calibration target 72 of the type shown in Figure 

5 IB. Other targets according to the invention, of course, can likewise be coupled to the 
object 70. The targets can be coupled to the object by any known means. For example, 
they can be molded onto, etched into, or printed on the surface of the object. By way of 
further example, decals embodying the targets can be glued, screwed or otherwise affixed 
to the object. Moreover, by way of still further example, calibration plates incorporating 

10 the targets can be placed on the object and held in place by friction. In addition to 

integrated circuit chips, the object can include any other objects to which a target can be 
coupled, such as printed circuit boards, electrical components, mechanical parts, 
containers, bottles, automotive parts, paper goods, etc. 

15 Figure 3 depicts a machine vision system 80 according to the invention for 

determining the reference point and orientation of an object 82 having coupled thereto a 
calibration target 84 according to the invention and, particularly, a four-region target of 
the type shown in Figure IB. The system 80 includes an image capture device 86 that 
generates an image of a scene including object 82. Although the device may be 

20 responsive to the visual spectrum, e.g., a conventional video camera or scanner, it may 
also be responsive to emissions (or reflections) in other spectra, e.g., infrared, gamma-ray, 
etc. Digital image data (or pixels) generated by the capturing device 86 represent, in the 
conventional manner, the image intensity (e.g., contrast, color, brightness) of each point in 
the field of view of the capturing device. 

25 

That digital image data is transmitted from capturing device 86 via a 
communications path 88 to an image analysis system 90. This can be a conventional 
digital data processor, or a vision processing system of the type commercially available 
from the assignee hereof, Cognex Corporation, as programmed in accord with the 
30 teachings hereof to determine the reference point and orientation of a target image. The 
image analysis system 90 may have one or more central processing units 92, main memory 
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94, input-output system 96, and disk drive (or other mass storage device) 98, all ot the 
conventional type. 



The system 90 and, more particularly, central processing unit 92, is configured by 
programming instructions according to teachings hereof for operation as illustrated in 
Figure 4 and described below. Those skilled in the art will appreciate that, in addition to 
implementation on a programmable digital data processor, the methods and apparatus 
taught herein can be implemented in special purpose hardware. 



Referring to Figure 4 there is shown a machine methodology according to the 
invention for interpreting an image of a target 84 to determine its reference point and 
orientation. The discussion that follows is particularly directed to identifying a four- 
region target of the type shown in Figure IB. Those skilled in the art will appreciate that 
these teachings can be readily applied to finding targets according to the invention, as well 
as to other targets having detectable linear edges that are oriented toward a reference 
location or reference point on the target, e.g., a prior art cross-shaped target. For 
convenience, in the discussion that follows, such linear edges are referred to as "adjoining 
edges," regardless of whether they are from calibration targets according to the invention 
or from prior art calibration targets. 

In step 100, an image of the target 84 (or of the target 84 and object 82) is 
generated, e.g., using image capture device 86, and input for machine vision analysis as 
discussed below. The image can be generated real time, retrieved from a storage device 
(such as storage device 98), or received from any other source. 

In steps 102 - 108, the method estimates the orientation of the target in the image 
using any of many alternative strategies. For example, as shown in step 102, the method 
determines the orientation by applying a conventional Hough line vision tool that finds the 
angle of edges discernable in the image. In instances where the target occupies the entire 
image, those lines will necessarily correspond to the adjoining edges. Where, on the other 
hand, the target occupies only a portion of the image, extraneous edges (e.g., from other 
targets) may be evident in the output of that tool. Although those extraneous edges can 
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generally be ignored, in instances where they skew the results, the image can be 
windowed so that the Hough vision tool is only applied to that portion that contains the 
target. Once the angles of the lines has been determined by the Hough line tool, the 
orientation of the image is determined from the predominant ones of those angles. 
Alternatively, the angle of the image can be determined by taking a histogram of the 
angles. 



The Hough vision tool used in step 102 may be of the conventional type known 
and commercially available for finding the angle of lines in image. A preferred such tool 
is the Cognex Line Finder, commercially available from the Assignee hereof, Cognex 
Corporation. 



An alternative to using a Hough vision tool is shown in step 106. There, the 
illustrated method determines the orientation of the target by applying a Sobel edge tool to 
the image to find the adjoining edges. Particularly, that tool generates a Sobel angle 
image that reveals the direction of edges in the image. As above, where the target 
occupies the entire image, the adjoining edges will be the only ones discerned by the 
Sobel edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 

The Sobel edge tool may be of the conventional type known and commercially 
available for finding lines in image. A preferred such tool is the Cognex Edge Detection 
tool, commercially available from the Assignee hereof, Cognex Corporation. 

Once the Sobel angle image is generated, in step 106, the orientation of the target 
in the image is determined by generating a histogram of the edge angle information; see, 
step 108. From that histogram, the target orientation can be determined by taking a one- 
dimensional correlation of that histogram with respect to a template histogram of a target 
oriented at 0° Where a Sobel magnitude image is generated, in addition to the Sobel 
angle image, such a histogram can be generated by counting the number of edges greater 
then a threshold length at each orientation. 
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As a still further alternative to applying a Hough vision tool or Sobel edge tool, the 
method contemplates obtaining the angle of orientation of the target from the user (or 
operator). To this end, the user may enter angle orientation information via a keyboard or 
other input device coupled with digital data processor 90. 

5 

In steps 110-1 18, the method determines the location, i.e., coordinates, of the 
target reference point in the image. Particularly, in step 1 10, the method can apply a 
Hough vision tool, as described above, to find the angle of lines discernable in the image. 
A conventional Hough vision tool determines, in addition to the angle of lines in an image, 
10 the distance of each line, e.g., from a central pixel. As above, where the target occupies 
the entire image, those lines will be the only ones discernable by the Sobel edge tool 
image. Where, on the other hand, the target occupies only a portion of the image, any 
extraneous edges can be ignored or windowed out. 

15 As above, the Hough vision tool used in step 104 may be of the conventional type 

known and commercially available for finding the angle and position of lines in image. 
Once again, a preferred such tool is the Cognex Line Finder, commercially available from 
the Assignee hereof, Cognex Corporation. Those skilled in the art will, of course, 
appreciate that steps 102 and 1 10 can be combined, such that a single application of the 

20 Hough vision tool provides sufficient information from which to determine both the 
orientation of the target in the image and its reference point. 

As an alternative to using a Hough vision tool, the method can apply a projection 
vision tool to the image in order to find the position of the lines discernable in the image; 

25 see, step 1 12. The projection tool, which maps the two-dimensional image of the target 
into a one-dimensional image, is applied along the axes defined by the edges in the image. 
As those skilled in the art will appreciate, the location of the edges can be discerned from 
by finding the peaks in the first derivatives of each of those projections. As above, where 
the target occupies the entire image, those lines will be the only lines discernable by the 

30 Sobel edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 
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The projection vision tool used in step 1 12 may be of the conventional type known 
and commercially available for mapping a two-dimensional image of the target into a one- 
dimensional image. A preferred such tool is that provided with the Cognex Caliper tool 
commercially available from the Assignee hereof, Cognex Corporation. 

5 

In step 1 14, the method uses the information generated in steps 1 10 and 1 12 to 
compute the location of the reference point, particularly, as the intersection of the lines 
found in those steps 1 10 and 112. 

10 As an alternative to using the Hough vision tool and the projection tool, the 

method can apply determine the location of the reference point by performing a binary or 
grey scale correlation on the image; see step 116. To this end, the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross-shaped pattern in the case of a target of the type shown in Figure IB. The use of 

15 correlation vision tools for this purpose is well known in the art. The template for such an 
operation is preferably generated artificially, although it can be generated from prior 
images of similar targets. 

As still another alternate to the Hough vision tool and the projection tool, the 
20 method can apply a grey-scale image registration using the sum of absolute differences 
metric between the image and a template; see step 118. To this end, the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross-shaped pattern in the case of a target of the type shown in Figure IB. The template 
for such an operation is preferably generated artificially, although it can be generated from 
25 prior images of similar targets. A preferred grey-scale image registration tool is disclosed 
in United States Patent No. 5,548,326, the teachings of which are incorporated herein by 
reference. 

Although steps 1 10 - 118 can be used to determine the approximate location of the 
30 reference point of the target in the image, the method utilizes optional steps 120 and 122 
to refine that estimate. These two steps are invoked one or more times (if at all) in order 
make that refinement. 
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In step 120, the method applies a conventional caliper vision tool to rind points in 
the image that define the adjoining edges of the regions. On the first invocation of step 
120, the method applies calipers along fifty points along each edge (though those skilled 
in the art will appreciate that other numbers of points can be used), beginning with points 

5 closest to the estimate of the reference point, as discerned in steps 1 10 - 118. The calipers 
are preferably applied a small distance away from the actual estimate of the reference 
point to avoid skewing the analysis due to possible misprinting of the target at that point, a 
missing pattern at that point (e.g., Figure 1C), or a too-high spatial frequency at that point 
(e.g., Figure IB). In step 120, the method then fits a line to the points found along each 

10 edge by the caliper tool, preferably, using a conventional least squares technique. 

In step 122, the method computes a refined location of the reference point as the 
intersection of the lines identified in step 120. Although conventional geometric 
calculations can be performed for this purpose, preferably, the reference point is computed 

15 using conventional least squares techniques. Preferably, when examining the image of a 
symmetric calibration target of the type shown in Figure IB, the method utilizes the same 
number of points on either side of (and closest to) the reference point for purposes of 
fitting each line. This minimizes the bias otherwise introduced by a conventional edge 
detection technique in finding edges that are defined only by dark-to-light (or light-to- 

20 dark) transitions. 

Calibration targets of the type shown in Figure IB are advantageously processed 
by a method according to the invention insofar as they further minimize bias otherwise 
introduced by a conventional edge detection techniques. In this regard, it will be 

25 appreciated that such bias is reduced by the fact that "opposing" adjoining edges (i.e., 
edges that oppose one another across the reference point) define straight linear segments 
that change polarity across the reference point. That is, those segments are defined by 
regions that transition — preferably, equally in magnitude - from light-to-dark one one 
side of the reference point, and from dark-to-light on the other side. This is true for all 

30 "symmetric" calibration targets according to the invention, i.e., targets in which opposing 
edges define straight linear segments that are opposite polarity on either side of the 
reference point. 
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On the second and subsequent invocations of step 120, the method specifically 
applies the caliper tool at points along the lines found in the previous iteration. 
Preferably, these points are at every pixel in the image that lies along the line, though, 
those skilled in the an will appreciate that few points can be used. The calipers can be 
5 applied orthogonal to the lines, though, in the preferred embodiment, they are applied 
along the grid defined by the image pixels. The caliper range decreases with each 
subsequent invocation of step 120. The method continues applying the calipers, beginning 
at the estimated center point until one of the four following situations occurs: no edge is 
found by the caliper applied at the sample point; more than one edge is found by the 

10 caliper and the highest scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the CONFUSION ^THRESHOLD); the distance 
between a computed edge point and the nominal line (computed from the previous 
invocation of step 120) is larger than a threshold (which threshold decreases with each 
subsequent invocation of step 120); or, the caliper extends outside of the image. As 

15 above, in step 120, the method then fits a line to the points found along each edge by the 
caliper tool, preferably, using a conventional least squares technique. 

In the second and subsequent invocations of step 122, the method computes a 
refined location of the reference point as the intersection of the lines identified in step 120. 
20 As above, although conventional geometric calculations can be performed for this 
purpose, preferably, the reference point is computed using conventional least squares 
techniques. 

■ Referring to Figure 5, there is also shown a machine methodology according to the 
25 invention for interpreting an image of a target 84 to determine its reference point and 
orientation. Particularly, in step 150, the method calls for generating an image of a target 
84 and, particularly, of a target according to the invention having two or more regions, 
each region being defined by at least two linear edges that are directed toward a reference 
point, and having at least one of the regions having a different imageable characteristic 
30 from an adjacent region. Step 152 can be effected in the manner described in connection 
with step 100 of Figure 4, or equivalents thereof. 
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In step 152, the method analyzes the image to generate an estimate of an 
orientation of the target in the image. Step 152 can be effected in the manner described in 
connection with steps 102 - 108 of Figure 4, or equivalents thereof. 



In step 154, the method analyzes the image to generate estimate of a location of the 
target's reference point. Step 154 can be effected in the manner described in connection 
with steps 1 10 - 118 of Figure 4, or equivalents thereof. 

In step 156, the method analyzes the image to refine its estimates of the location of 
the reference point in the image and the orientation of the target in the image. Step 156 
can be effected in the manner described in connection with steps 120 - 122 of Figure 4, or 
equivalents thereof. 



Calibration target and methods for analysis according to the invention are 
advantageous over prior art targets and methods insofar as they are magnification 
invariant. By analyzing the adjoining edges of targets, methods according to the invention 
insure reliance on features (to wit, regions) that retain the same imageable appearance 
regardless of magnification. This is in contrast to prior art targets and methods, which rely 
on individual lines (or dots) to define calibrating features. As noted above, the imaging 
appearances of such lines and dots change with varying degrees of magnification. Even 
the prior art methods that analyze checkerboard targets rely on analysis of corners, which 
are not magnification invariant. 



The magnification invariance of targets and methods according to the present 
invention is illustrated in Figures 6 A - 6C. Referring to Figure 6 A, there is shown an 
imaging setup wherein camera 200 images a target 202 (on object 204) from a height x. 
An image generated by camera 200 is displayed on monitor 206 of workstation 208. 
Referring to Figure 6B, there is shown an identical imaging setup, except insofar as a 
camera (of identical magnification) images a target (of identical size) from a greater 
height, x\ Likewise, Figure 6C shows an identical imaging setup, except insofar as a 
camera (again, of identical magnification) images a target (again, of identical size) from a 
still greater height, x". Comparing monitor images depicted in Figures 6A - 6C, it is seen 
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that the images of the targets, generated by the cameras and displayed on the monitors, are 
identical in appearance regardless of the relative heights (x, x\ and x") of the camera. This 
magnfication invariance results from the fact that the target has the same appearance 
regardless of height (or equivalently, magnification), within specified limits thereof. 
Those skilled in the art will appreciate that those limits are greater than the analagous 
limits for prior art targets (e.g., cross hairs, parallel stripes, etc.). 

Still another advantage of calibration targets and methods according to the 
invention is that they permits angular orientation to be determined throughout a full 360° 
range. With reference to Figure ID, for example, the relative positions of the regions can 
be used to determine the overall orientation of the target, i.e., whether it is rotated 0°, 90°, 
180°, or 270°. This information can be combined with a determination of relative 
orientation made by analysis of the adjoining edges as discussed above to determine the 
precise position of the target. 
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APPENDIX I 



Patent Application for 



MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION 



Vision Tool Description 



THE FOLLOWING APPENDIX IS NOT BELIEVED TO BE NECESSARY FOR 
ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED AND 
CLAIMED IN THE ACCOMPANYING APPLICATION. 
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Edge Detection Tool 

This chapter describes the Edge Detection tool, which includes edge detection ana peak 
detection. 

Edge detection takes an input image ana produces two output images: an image of the 
edge magnitude of each input pixel and an image ot the edge angle of each input pixel. 
When combined with other Cognex software tools, tne information produced by edge 
detection can be used to locate objects within an image. Edge pixel information can be 
used to detect the rotation of an obfect or detects such as scratches, cracks, or particles. 
You can aiso gauge now snarpty an image is focusea with edge pixel information. 

Peak detection is useful in any application m which knowledge of tne local maximum veJuee 
in a two-dimensional soace is useful. Peak detection takes an input image and produces 
an output image containing only those pixels in the input image with higher values than 
neighboring pixels. A typical input image to peak detection is an edge magnitude image, 
the output image contains only the highest magnitude edge pixels. The edge detection 
function can optionally use peak detection to postprocess the edge detection results so 
that only the strongest eages remain. 

This chapter has nine sections as follows: 

An Overview of Edge Detection describes the goais of edge detection, defines an edge 
pixel, and describes now the Edge Detection tool finds edge pixels. This section also 
explains the two properties of a pixel that are calculated by the Edge Detection tool, eage 
magnitude and angle, and ends with a sample application. 

Using Edge Detection describes, in general terms, the interface to the Edge Detection toot. 
Tnis section contains a aiscussion of tne compression tables that affect tne amount of 
memory used by the edge detector. 

Edge Detection Enumerations and Data Structures descrioes the data structures and 
enumerations that the edge detection functions use. Types and data structures mat 
support peak detection are discussed in Peak Detection Enumeration Types and Data 
Structures below. 
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Edge Detection Functions describes the functions that mplement eage detection. 

Maximizing the Performance of VC2 Vision Coprocessor lists tne edge detection parameter 
settings that you should avoid if you want to maximize the performance of tne VC2 vision 
coprocessor during edge detection. 

Overview of Peak Detection defines DeaK pixel and gives examples of some applications 
of peak dot ©ebon 

Using Peak Detection describes the interlace to peak detection. 

Peak Detecvon Enumeration Types and Data Structures describes the peak detection 
enumerations a no aata structures. 

Peak Detection functions describes the o«ak detection functions. 
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Some Useful Definitions 



edge pixel 

edge detector 
compression table* 



Sobel operators 

horizontal edge 
component 

vertical edge component 
edge magnitude 
edge angle 



peak pixel 

peak detection 

8-polnt neighborhood 

4 -point neighborhood 
2 -point neighborhood 



Pixel that has a different value from one or more of its 
eight neignooring pixels. 

Operator that finds edge pixelc in an imago. 

Two tables thai are created during edge detection 
initializanon: the edge compression table and the 
magnitude compression table. These tables ere used by 
the Edge Detection tool to map data into a smaller range 
of values. 

Two 3x3 operators mat the Edge Detection tool uses to 
locate edge pixels in an image. 

Value returned by the horizontal Sobel operator when it is 
aoplied to a pixel in an image. 

Value returned by the vertical Sobel operator when it ts 
applied to a pixel in an image. 

value mat increases as the difference in grey levels 
between neighboring pixels increases. 

Orientation of an edge, with respect to the x*axis of the 
image, at an edge pixel. The Edge Detection tool defines 
me angle of a pixel as arctan(wn). wnere vis the vertical 
eage component and h o the nonzontal edge component 
of the pixel. 

Pixel with a value greater than or equal to some or alt of 
its neighboring pixels' values. 

Finding some or alt peek pixels in an image. 

Pixels mat are horizontally, vertically, and diagonally 
adjacent to a pixel. 

Pixels that are horizontally and vertically adjacent to a 
pixel. 

Neighbors of a pixel along a single axis. Peak detection 
supports a horizontal and a vertical 2-pomt neighborhood 
operator. 
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•ymmetrftc peak 
•symmetric peak 

aUnms peak 
elngta-axla peak 
plataau 



Peak whose pixel value is greater tnsn or equaJ to tne 
values or each of its neighbors. 

Peak wrvose pixel value is greater than the values of its left 
and lower neighbors and greater than or equal to the 
values of lis right and upper neignoors. 

Pixel mat satisfies the peak definition (symmetric or 
asymmetric) along at) axes in its neighborhood. 

Pixel that satisfies the oeak definition (symmetric or 
eaymmetno) along et least one axis m its neighborhood. 

In oeak detection, a region of contiguous pixels of the 
same value. 
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An Overview of Edge Detection 

The Edge Detection tool finds edge pixels in an image. This overview of edge detection 
oontainc tho following descriptions: 

• Definition of the problem of edge detection 

• Overview of the tasks that the Edge Detection tooi performs 

• Definition of edge pixel 

• Description of the Sotoe! operator, wnich is used by the Edge Detection tool to locate 
edge pixels 

Definitions of tho angle and magnitude of an aoge pixel, and deocrlpuone of tha two 
images produced by the Edge Detection tool: the magnitude image and the angle 
image 

• Description of edge detection preprocessing 

• Description of edge detection postprocessing 

• Description of a sample application that uses both the magnitude and the angle 
images to create an angle "signature* of a shape 

Edge Detection 

Tne Edge Detection tool locates edges and determines each edge's angle and magnitude. 

An edge occurs wherever adjacent areas of an image have different grey values. Using this 
definition the triangle in Figure 66 has a single edge. 




Figure 66. A figure with an edge 
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The definition of edge can be further refined es the directed border between grey areas of 
an image, where direction is defined as a vector normal to the edge. Using this definition, 
the triangle in Figure 67 has three directional edges, reprafiantAd by tha thr«a vectors. Tno 
angle of the edge is the counterclockwise angle of the vector normal to the edge, with 
respect to tne horizontal axis of tne image. See the section Angle Image on oaoe 1 81 for a 
discussion of edge angle. 




Figure 67. Vectors indicating mo eage angles of a triangle 

Along with an angle, an edge has a magnitude, which reflects the amount of the difference 
between grey levels on either side of the edge. As the difference in grey levels increases, 
the magnitude increases. See the section The Magnitude Image on page 178 for a 
discussion of edge magnitude. 

The triangle in Figure 68 has the same three edges as the triangle in Figure 67: however, 
these edges are of lower magnitude. This lower magnitude is Illustrated by the snortenea 
length or tne three direction vectors. 




Figure 68. A triangle with low-magnitude edges 
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Edge Detection Tool 



Edges are actually located pixel by pixel. Figure 69 contains a magnified view of a portion 
of an edge in an Image. Each cell in the figure represents the grey level of a single pixel. 
Notice that on this -microscopic- level, there are many edoes between pixels in whicn the 
grey levels on either side of the edge differ by a few percentage points. 
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Figure 60. Grey levels m a magnified portion of an edge 

The Edge Detection tool locates edges in an image by identifying each pixel that has a 
different value from one of its neighboring pixels. It calculates the angle ana magnitude of 
eacn edge pixel. It also orowdes a means of classifying edges, so thai low-magnitude 
edges are not reported. 
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Edge Pixels 

A pixel is an edge pixel if it hae a different value from at leaet one of ite eight neighboring 
pixels ana is not on the border of an image. In Figure 70. the four snaaea pixels are edge 
pixels. 
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Figure 70. The shaded pixels ere edge pixels 

Figure 71 contains a grid representing grey levels in an image with the highest magnitude 
e<Jge pixels shaded. Notice that border pixels (pixels along the edge of the image) are not 
edge pixels. 



0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 




3 


0 


0 


0 


0 


0 


0 


0 






4 


4 


2&44J. 




0 


0 


0 


0 


0 






4 


4 


4 


4 


0 


0 


0 




Hi 


4 


4 


4 


4 


4 


4 


0 






4 


4 


4 


4 


4 


4 


4 


4 


3 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 



Figure 71. Highest magnitude edge pixels in en imege 
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The Edge Detection Operators 

The Edge Detection tool finds edge pixels using the Sabot 3x3 neighborhood operators. 
During edge detection, the Sobel operators are applied to each pixel In an input image. 
This operation produces two values tor each pixel. One value represents the vertical edge 
component tor the pixel; the other value represents the Horizontal edge component tot the 
pixel. These two values are then used to compute the edoe magnitude and edge angle of 
the pixel. 

Sobe! edge detection uses two 3x3 neighborhood operators to locate edge pixels In an 
image. The horizontal Sobel operator detects the horizontal (x) edge component for a pixel. 
Tn« vertical Sooei operator aetects tne vertical (v) eoge comDoneni for a pixel. Figure 72 
snows the Sobel operators. 
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Figure 72. Sobel operators 

The Sobel operator works only on pixels with eight neighboring pixels. When the Sobel 
operator is applied to a pixel on the border of an image, tne result is defined to be 0 
because there are not enougn neighboring pixels to calculate an edge value. 

The horizontal and vertical Sobel operators are 3x3 linear operators. These operators 
compute a value (or a pixel (x.y) in the following way: 

• The grey level of tne pixel (x.y) is multiplied by the value in the center of the 3x3 linear 
operator. 

• The grey level of the neighboring pixel (x-7.y-7) is multiplied by the first value in the 
top row of the 3x3 linear operator. 

• The grey level of the neighboring pixel (x.y-1) is multiplied by the second value in the 
top row of the 3x3 linear operator. 

• Thic continues unta the produot of eeoh pair of veluee ie calculated. 

• These products are then summed to produce the output. For the Sobel operator the 
value is interpretea as horizontal or vertical edge component or tne pixel. 
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If the result of applying at least one of the Sobel operators to a pixel is nonzero, that pixel 
is an edge pixel. 
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Figure 73, Applying the Sobel operator 

In Figure 73, the horizontal Sobel operator is applied to the shaded pixei in the image. The 
resultant edge pixel value is 6. which is calculated as follows: . 
-1.U0-U12 + -2 U0 2t2 3t.. 12 + 0 3+1-3 = 6 

A/ote:The range of pixel values in Figure 73 and in most examples In this document is much 
analler than the typical range of pixel values in an image. This is done to simplify the 
examples. 

Figure 74 shows fill the vertical and horizontal edge component values for a email image. 
The upper grid is the image; the values represent the grey levels of each pixel. The grid to 
the lower left Of the image contains horizontal edge component values for the image. Each 
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cell in the grid contains the result of applying the Sobel horizontal operator to the 
corresponding input Image pixel. The grid to the lower right contains the vertical eage 
components tor the image, it is comDuted using the Sobei verocei operator. 
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Vertical edge component values 



Figure 74. Horizontal ana vertical edge component values computed 
with the Sobel operators 

The horizontal and vertical edge component values computed with the Sobel operator are 
not returned by the edge detection functions. They are used to compute tne edge 
magnitude ana eage angie or each pixel in tne image. 
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The Output images 



Edgo magnitudes are stored in an output image thai chows in© edge magnitude of each 
corresponding pixel in the input image. Edge angles are stored in another output image 
that shows the edge angle of each pixel. You can choose which output images to create: 
magnitude, angle, or both. 

These two images are described in this section. 

Jbm Magnitude) \mmqm 

Tne edge magnitude for a given pixel is a function of the horizontal and vertical edge 
components of the pixel. If xis the horizontal edge component value and y is the vertical 
edge component value tor some pixel, men tne eage magnitude. M. for mar pixel is aerinea 
as follows: 

M = V* + y 

This formula suggests a geometric interpretation of the data: if xis the horizontal edge 
c o mponent and y is the vertical edge component, the magnitude is a vector from tne origin 
of a two-dimensional coordinate system to the point (x,y). Figure 75 shows the geometric 
relationship between the edge magnitude end the two edge oomponenta. 



Vertical edge 
component 




Edge magnitude 



Horizontal edge 
component 



Figure 76. Geometric representation of edgo magnitude 

Note: For imoiementation-soecific reasons, the Edge Detection tool uses the formula 
described above to compute edge magnitude and then scales the magnitude upward 
by approximately 16%. 
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You can choose the numDer ol bits that can be used to express magnitude. Because 
magnitude values, computed with vertical and horizontal edge component values, can 
exceed this number of bits, magnitude values are compressed to an integer value that can 
oe expressed in the number of bits you have specified. You can control how magnitude 
values are comDressed: you can choose a method suoDlied by the Edae Detection tool, 
such as a logarithmic or linear map, or you can supply a magnitude compression map of 
your own. For descriptions of the maps supplied by the Edge Detection tool and for 
information on using your own map. see tne section Two Compression Tabieson page 192. 

The size and depth of the magnitude image are the same as that of the input image. Since 
all border pixels have horizontal and vertical edge pixel values of 0, the border of the 
magnitude image contains ail zeros. You can display an edge magnitude image: higher 
magnitude edges are brighter. 

Optionally, edge detection can return the numDer of edge pixels in tne input image and the 
sum of ail edge magnitude values. By dividing the sum of all edge magnitude values by the 
size of the image, you can compute tne average edge magnitude of an image. If you have 
two Images of the same scene, you can determine which image is in sharper focus by 
comparing average edge magnitudes of the two images: the image with the higher average 
magnitude is in sharper focus. 
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Figure 76 contains an image, the image's horizontal and vertical edge components 
computed with Sobei operators, and the output image containing the edge magnitude 
values lor each pixel. The edge pixels with the highest magnitude, representing the actual 
edge, are shaded. 
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Ftgure 76. Edge magnitude values computed with me Sooei operators 

Angte Imago 

The edge detection functions produce an imago containing tne angle of each edge pixel. 
If /Is the horizontal edge component value and yis the vertical edge component value, the 
edge angle ie the counterclockwise angle between the horizontal axis and tno magnitude 
vector. 

Figure 77 snows tne geometric relationship between the edge angle, tne edge magnitude, 
and the two edge components. In this figure, M is the magnitude vector and 6 is me edge 
angle. 
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Ftgure 77. Geometric interpretation of edge angie 

Edge angles are stored In the angle image in binary form. They are represented as 
fixed-point numbers with tne decimal point to the left ot tne most significant bit and are 
interpreted as tractions of 360°. You can choose the maximum numner of bits available to 
express an angle; this value must be less than or equal to 8. If n is the maximum number 
of bits used to express angle, and x is the binary representation of an angle, you can 
convert xto degree notation by using the following formula: 



anglc(x) = — jj- 

Note that with a 6-bit representation, angles can be expressed to the nearest 5.626°; with 
a 7-blt representation, an angle can be expressed to the nearest 2.8°: with an 8-bit 
representation, an angle can be expressed to the nearest 1.4°. 
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The edge angle is computed using the arc tangent function if *is the horizontal edge 
component end y is the vertical edge component tor a pixel, the eage angle for that pixel 
10 celoulated ac shown in Table 3. 
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Table 3. Formulae tor computing edge angles 
The computation is designed so that an edge angle e is in tne following range: 

0°£6< 360° 
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Figure 78 contains six rows. Each row contains (from left to nght) an image with the central 
pixel shaded, the vertical and horizontal edge component value oi the shaded pixel, the 
image with a vector super rmoocod showing th* edge angle of the contra! pixel, and the 
tormula used to compute me edge angle. Notice that the equations defining me edge angle 
value are designed so that the vector always ooints to the briohte; side of the edge. 
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Figure 78. Computing tne angle of various edgos 
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By setnng a flag in the edge detection parameters, you can specify thai you want angles 
computed m the range 0° to 160° in9teaa of the default range, 0° to 360°. When you select 
tne smaller range, an edge angle nthat is greater than 1B0° maps to edge angle (n- 180°). 
For instance, 300" maps to 120°, and 270° map9 to 90°. 

Use The reduced angle range when you want tne same edge angle results regardless of 
the polarity of the image. As shown in Figure 79. when the range of angles is restricted to 
the range 0° to 180°, the same edge angles are computed for a black object on a white 
background as tor a white object on a black background. When the range of angles is 0° 
to 360° (also Figure 79) this is not the case: reversing the polarity of The image reverses the 
edge pixel angle. 




Figure 79. Effect of reversed polarity on edge angles for the two ranges 

The binary representation of an angle in tne range 0° to 1 80° is similar to the representation 
of an angle in the range 0° to 360°. It n is the maximum number of bits used to express 
angle and xls the binary representation ol an angle in the range 0° to 180°, you can convert 
xto degree notation by applying the following formula: 

I80x 

angle(x) = — 
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Edge Detection Preprocessing 

If you supply a pixel map. the edge detection function usee it to preproceaa your input 
image before It calculates the horizontal and vertical edge components. For a description 
of Dixet mapping, aee Chapter 3. Pixel Mapping. 

Edge Detection Postprocessing 

Although edge angles are defined for all pixels in an image, they are significant only at the 
pixels with me greatest edge magnitudes. There are many edges between pixels where the 
grey levels on either aide of the edge differ by only a few percent. These ineignif ioant edge 
pixels, due to frame grabber noise and random fluctuations in pixel values, can clutter your 
edge magnitude image and make it difficult to evaluate tno data in that imago. Aioo edge 
magnitude values usually increase gradually in a "smear* as they approach an edge and 
then decrease ac the edge ic crocced. You might want to cnorpon the magnitude image by 
filtering out some edge pixels that are near, but not at. an edge. 

The Edge Detection tool supplies three postprocessing methods. Two of these methods. 
setup and run-time thresholding, set the lowest edge magnitude pixels to 0. The third 
method, pea* detection, takes an edge magnitude image ana processes it so mat. tor any 
small pixel neighborhood, only the highest edge magnitude pixels remain and the other 
pixels are set to o. See tne section Pea* oexecvon on en Eage Magnitude Image on 
page 229 for a complete discussion of this method. 

After postprocessing, edge angles corresponding to zero edge magnitude pixels are also 
set to 0 unless you have specified run-time thresholding but not peak detection. In this 
case, edge angles corresponding to 2ero edge magnitude pixels can be nonzero, but 
should be considered undefined. 

£#tup Thresholding 

You can supply a setup time rejection percentage. This is a value n that represents a 
percentage of the hignest possible edge magnitude value. With this method, any 
magnitude values in the first n percent of all possible values are forced to 0. For example, 
if you provide 6 bits to express magnitude values and specify that the lowest 5 percent of 
magnitude values are forced to 0, then any magnitude below 3 is forced to 0 since 5 
percent of 63 (truncated) is 3, 

Run-Time Thresholding 

You can supply a run-time rejection percentage. This is a value n that represents a 
percentage of the highest actuai edge magnitude value in a magnitude image. With this 
method, magnitude values in the first n percent of the magnitude values in a specific image 
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are forced to 0. For example, if the highest magnitude value tn a particular image is 51 . end 
if you specify that the lowest 5 percent of the run-time magnitude values are forced to 0, 
then any maonitude below 2 is forced to 0. since 5 percent ot 51 (truncated) is 2. 

Using a setup rejection percentage is taster than using a run-time rejection percentage, 
since the values that are forced to 0 are computed once, before any magnitude images are 
created. However, the run-time method is more flexible and robust. If you are producing 
magnitude images or the same scene dui under varying lighting conditions, you snouta use 
a run-time rejection percentage since the proportion of magnitude values forced to 0 
remains stable as the range of magnitude values changes (due to varying illumination). 

Postprocessing with Peak Detection on a Magnitude triage 

Peak detection te a method of filtering en image ao that only ooak ptxots remain and other 
pixels are set to 0. Peak pixels nave a value greater than or equal to some or all of their 
neighboring pixel©' values. 

You can set a flag in the edge detection parameters structure that instructs the edge 
detection function to run peak detection on the magnitude image during postprocessing. 
All nonpeak pixels are filtered out of the magnitude image and the edge angle image before 
they are returned. When you specify this, you must supply peak detection parameters in 
the edge detection parameter structure. 

The edge detection function does peak detection postprocessing after it has done setup 
and run-time thresholding. There will be a 2-pixel-wlde border of zeroo in a peak-delected 
magnitude image. 

The section Peak Detection on an Edge Magnitude Image on page 229 discusses, in detail, 
the benefits of using peak detection on an edge magnitude image. This section also 
describes the proper way to parameterize peak detection so that it filters edge magnitude 
pixels correctly. However, you do not need a detailed knowledge of peak detection to use 
it with edge detection. Suggested peak detection parameters are supplied in the section 
Data Structure cip_edgej?arams on page 199. 

Figure 80 contains an image, its edge magnitude image, and the image that results from 
peak detection. The edge magnitude image contains a smeared edge, which is sharpened 
by peak detection. 
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Figure 80. Peak dejecting an edge magnitude image to determine actual 
edge 
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Sample Application: Angle Histogram 

You con uee tne magnitude and angle image* of an object to oroate on angJo histogram of 
that object. This is a table mat assigns a value n to each edge angle 6 if there are nedge 
pixalc (above a given magnitudo throshold) whoaa edge angle ts 6. 

The angle histogram supplies a useful •signature* of an object's shape. You can use this 
signature to gauge similarity among objects, to detect rotations of an object, and to detect 
flaws in an object. 

Once you have created edge magnitude and edge angle images for an object, you can 
create and display on angle histogram ae followe: 

void anffle_histograjn < res , threshold ) 

cip_edge_results Tes; /• results structure pointer •/ 

int~threahold; /* significant edge threshold V 

i 

int i # j,mag; 
int hist{256) ; 

cu_clear(hist, sixeof (hiat) ) ; 

tor (1=0; i<ros->mag_lmg_p-> width; i**) 

for (j=0; j<res->mag_iing.p->height; j**) I 

Biag=res->mag_ling_p->get (r«3-»mag_iing_p, i , j > ; 
if (mag > threshold) 

hist(res->ang_img_j>->get (res->ang_img_ p.i. jH**; 

) 

cgx.hi a togr «n ( e Aq.iraAff ft . hi c C . 6 5 ) : 

caa_di splay ( ) ; 

return; 

) 
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Figure 81 contains four shapes; each shape Is paired with its angle histogram. Note that 
the square and the plus sign have identical angle histograms. The angle histogram of the 
circle is flat because there is no dominant angle along the eage of a cirelo. 




0 90 1B0 270 360 
Angle 




Number of 
pixels 



0 90 180 270 360 
Angle 




0 90 180 270 360 
Angle 




Angle 



Figure ei. Assorwa snapes wim their angle histograms 
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Figure 82 demonstrates tne effect of rotation on the angle histograms of a plus sign. The 
histogram shifts as the plus sign rotates. 
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The Caliper Tool 

This section defines the Caliper Tool and describes in general terms how it works. The 
concepts introduced in this section are described in detail in tutor section* 

The Purpose of the Caliper Tool 

The Caliper Tool is a tool for locating edges and edge pairs in an image. The edge of an 
object in an image is a change in grey value from dark to light or light to dark. This change 
may span caveral pixelc. Tho Caliper Tool provides methods for ignoring edgaa that are 
caused by noise or that are not of interest for a particular application. 

The Caliper Tool is modeled on the mechanical engineer's caliper, a precision device for 
measuring distances. You specify the separation between the caliper 'jaws.* and the 
Caliper Tool searches an area you specify for edge pairs separated by that distance. You 
can aJso search for Individual edges when you know their approximate location in an 
image. This is similar to using a caliper to measure depth. 

A typioai uee tor the Caliper Tool 1a part inapection on an assembly lino. For example, in 
integrated circuit lead inspection, a part is moved in front of a camera to an approximately 
known location (the expected location). You use the Caliper Tool to determine the exact 
location of the left edge of the pan by searching for a single light to dark edge at the 
expected location (see Figure 1 03) . The edge that is closest to the expected location is the 
left edge of tne pan. 



Search |mage 




Figure 103. The Caliper Tool is used to locate the left edge of the part. 



43 



WO 98/18117 



PCT/US97/18268 



4 Caliper Tool 



Once you find the left edge of the part, you can predict where the leads should tie within 
the image to specify a window m which to look lor leads (see Figure 1 04). You can then use 
the Caliper Tool to search tor edge pairs consisting of a dark to light transition followed bv 
a light to dark transition, ana separated by the expected lead width. With information 
calculated by the Calioer Tool, you can compare measured lead width and location to 
expected lead width and location to make an accept/reject decision about the pan. 




Figure 104. The Caliper Toot can be used to measure lead width and 
load spacing on an integrated circuit. 



How the Caliper Tool Works 

The Caliper Tool uses protection to map a two-dimensional window of an image (the caliper 
wtndovj) into a one-aimensional image. Projection collapses an image by summing the 
pixels in tne direction or tne projection, wnicn tends to ampury edges in mat direction. 
Figure 105 shows an image, a caliper window, the one-directional image that results from 
projection, end a graph of the pixel grey values in the one-dimensional image. 
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Image Caliper window 




One-dim enaional 
intensity image 



First edge Second edoe 



Figure 105. Overview of the Caliper Tool 

You may specify that the Caliper Tool perform pure/ mapping betore projection. Pixel 
mapping can be used to filter an image, attenuate or amplify a range of grey values, and 
otherwiee map pixel value*. See Chapter 3. Pixel Mapping, in the image Proceeding 
manual for a complete description of pixel mapping. 

After projection, an edge fitter is applied to the one-dimensional image to further enhance 
edge information and to smooth the image by eliminating minor grey value changes 
between neighboring pixels that are most likely caused by noise. The edge filter produces 



45 



WO 98/18117 



PCT/US97/18268 



4 Caliper Tool 



a first derivative image (the most rapid changes in grey value in the projected image result 
in the highest oeaks in the filtered image). Figure 106 shows tne one-dimensional image 
from Figure 105. an image generated toy the edge filter, ana a graph of that image. 




First edge Second edge 



Figure 106. An eage finer is applied to me one*atmenstonal image that 
results from projection 

After the fitter operation, edge detection is performed on the filtered image. Edge detection 
is a method of evaluating peaks in the filtered image and ignoring ©Ogee that are not of 
interest, such as eoges representing noise in the image. Edge detection applies geometric 
constraints that you specify to each edge or edge pair found in the image. Applying 
geometric constraints provides a way to limit the number oi edges to evaluate by assigning 
a score to each edge or eage pair cased on such factors as the expected location of the 
edge or edge pair, the distance between edges in an edge pair, and the minimum grey 
level dlflerence oetween pixels on each side of the edge. 

The Caliper Tool returns as many results as you have specified in the cclp^params data 
structure and computes the geometric average of tne score of eacn geometric constraint 
that you specify. For edges or edge pairs whose score equals or exceeds the accept 
threshold, the Caliper Tool returns information such as the score and location ot the edge 
or edge pair and the measured distance between edges in an edge pair. 

The controlling variables of an edge or edge pair search define a caliper, and include such 
information as the dimensions of the search window and the edge detection scoring 
parameters. The type of tne data structure mat defines a caliper is ccfcp_caUper 
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The Caliper Window and Projection 

This section describes how you specify the caliper window, which js trie subset of an image 
in which to locate edges or edge pairs. It men describes projection, which is ina mapping 
of the caliper window into a one-dimensional image. Finally it describes the edge titter, 
which enhances edges in the one-dimensional image. 



Caliper Window 

The caliper windowm ihe portion of an imago in which the Caliper Tool coarcnoc for edges. 
It is defined by the search length and projection length, which are the widtn and height 
respectively, of the window in an image that will be projected into a one-dtmansional image. 
The caliper window is illustrated In Figure 107. 
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Figure 107. The Caliper Tool window 

The pixel in the center of tne caliper winnow is called the application point This is the point 
in the run-time image at which you want io locate edges. 



Caliper Window Rotation 

For many applications edge information is needed at more than one angle in an image. For 
example, to inspect a rectangular pan, you may want to measure the dimensions of the part 
by oroiectino the image first at 0° to measure length, and then at 90° to measure width. The 
caliper window can be oriented at any angle to locate edge9 in an image. Angles increase 
in the clockwise direction from the horizontal axis. 

Figure 1 08 shows how changing the caliper window angle affects the projection and edge 
search directions at 90* rotations. The Caliper Tool runs faster for 90* f otanons man it does 
for arbitrary rotations. 
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Figure 705. 7he caliper window ai 90° rotations 



Skewed Wfndow Projection 



Figure 109 shows a *15° caliper window with window rotation disabled. In this case, the 
Caliper Tool creates a "skewed* winaow ena projects it along the angle of skew into a one* 
dimensional image. 
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Figure TOO. Skewed window projection 

When window rotation is disabled, you specify how the skewed window will be projected: 
with or without interpolation. Interpolation increases accuracy at tne expense of execution 
speed. 

When you use a skewed window without interpolation for projection the pixels in the source 
image ere summed along the angle of skew as shown in Figure 1 10. Each pixel in the 
two-dimensional image contributes to one pixel in the one-dimensional image. In 
Figure 110. thoce pixels containing the number 1 contribute to the destination pixel 
containing the number 1 , and so on. 
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Figure 1 10. Skewed window projection without interpolation 

When you use a skewed window with interpolation for projection, the skewed window is 
transformed into a rectangular window oefore projection. The transformation is performed 
by cipjrancf orm(), which uses several neighboring pixels from the two-dimensional 
image to calculate the pixel value tor the one-dimensional image. See Chapter 2. Basic 
Image Processing, in the Image Processing manual for a complete Description of The 
function olp_tr«n«form(). 
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Rotated Window Projection 

Figure 111 ehowc a -16° caliper window with window rotation onabiod. In this caco the 
Caliper Tool rotates the two-dimensional image and projects it into a one-dimensional 
image. 




Choosing the Projection Type 

The choice ot skewed or rotated projection depends on the angle of the edge or edge pair 
of interest in the image and the remaining content of the image. It the rotation angle is close 
to 90° or it a skewed image will contain enough of the edge or edge pair of interest to 
generate strong edges, you can use skewed projection to speed program execution. 
Figure 1 12 shows an image, a caliper window and the one-directional image that results 
from projection. The skewed window contains enough edge information, so skewed 
projection may be used. 
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Image 




Figure 112. Skewed projection may bo used 

Figure 1 13a snows a caliper window wnere tne cages of interest are at 15° ana wtnaow 
rotation is disabled A window large enough to enclose these edges would also include 
much of the dark area in tne image, wnicti in some cases would result in a one-dimensional 
image where the edges are obscured. 

Figure 1 13b shows a caliper window at -15 c in the same image with window rotation 
enabled. Only the edges of interest are included in this window. 
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figure 7 13. Skewed protection (a) and rotated projection (b) 



Caliper Tool Optimization 



When setting up the variables for a caliper, you specify the following types of optimization: 

• Space optimization: The Caliper Tool is optimized for minimum memory use at the 
expense of execution speed. 

• Speed optimization The Caliper Tool will use a number of internal techniques to 
improve execution speed at tne expense of increased memory use. 

• Default optimization: The Caliper Tool will weigh space and speed equally, using 
slightly moro ©pac© than if you cpocify cpao* optimization and clighUy elowor 
execution speed man if you specify speed optimizalion. 
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The Edge Filter 

After projection, an eage filter is run on the resulting one-dimensional image. The edge fitter 
accentuates edges in the image and produces a firterod image. The paaks in this image 
indicate strong edges. 

The two parameters of the edge filter are its size and leniency. Edge filter size is the number 
of pixels to each side of the edge to consider in the evaluation. Edge filter leniency is the 
number of pixels at an edge to ignore between light and dark sides of the edge (leniency 
is described in furtner detail below). Figure 1 u illustrates an edge filter. The edge filter is 
positioned over the leftmost ptxe! in the one-dimonsional image where it will entirely fit in 
that image. Pixel values to the left of the leniency region are summea and subtracted from 
iha ttum of the pixel veluee to the right of the leniency region: in mia example, (O-rS)-(O-O) 
yields 5 for the edge filter image. 
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Figure 1 14. Tho oage filter to ponitionect in the one-dimenetonal imago. 

As shown in Figure 115, the edge filter is men applied at each successive pixel in the 
one-dimensionaf image until the edge filter no longer fits entirely in the image. At each pixel 
position, the pixel values to the left of the leniency region are summed and subtracted from 
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the sum of the pixel values to trie ngnt ot the leniency region ana the result is written to the 
edge filter image. The resulting image will be 2 * size ♦ leniency - 1 smaller than the 
one-dtmensional image. 
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Figure 1 15. The edge filter is applied at each successive pixel until the 
edge filter no longer fits entirety in tne image. 

Figure 1 1 6 shows a graph of the one-dimensional image and the edge filter image from 
Figure 115. Peak position is calculated to sud pixel accuracy. 



55 



WO 98/18117 



PCT/US97/18268 



4 Caliper Tool 




image 



Figure 1 16. Edge f titer image graph vs. one-dimensional image graph 

The graph in Figure 116 represents an eoge niter apolled to an (deal image; few 
applications would result in as ideal a graph. All Images contain some degree of noise, and 
a typical edge in an Image can be several pixels wide. The next two sections provtae 
guidelines for optimizing the edge filter by choosing appropnate values for size and 
leniency. 

Choosing Edge Filter Size 

Eage filter size specifies tne numoer of pixels on eitner side of an edge to consider in an 
edge evaluation. Size should usually oe greater than 1 Because noise in an image causes 
grey level differences between neighboring pixels. Figure 1 17 snows a graph Of a 
one-dimensional image of a light band on a dark background, and a graph of an edge fitter 
image where a size of 1 was used. Because there is noise in the image, many more peaks 
exist than are of interest. 
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Edge filter peaks 



Figure 117, An eage filter size of 1 i$ typically too small. 

increasing size tenas to smootn tne eage niter image oecauae summing several pixels on 
either side of an edge tends to reduce the effects of noise. Figure 118 shows an image 
similar to the one in Figure 117 where a size of four was used. The only two peaks in me 
image are the peaks of interest. Size is typically between 2 and 5 inclusive. 
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Edge filter peaks 



Figure 1 1 ft Using four for edge filter size reduces the effects of noise on 
the edge tiller. 

Choosing Edge Filter Leniency 

Edge filter leniency specifies a region of pixels to ignore at an eage In an image. Due to 
slight changes in edge or edge pair position or orientation, a pixel on the edge of a feature 
may not always have the same grey value. By setnng a leniency zone between the tight and 
dark areas of a feature, the effects of slight changes in feature location on filter results are 
minimized. 

Figure 119 shows a graph of a one-dimensional image with two edges that span two pixels 
eacn, and graphs of the output of two edge filters. The first edge filter has a leniency of 0, 
the second edge filter has a leniency of 2. The edge fitter with a leniency of 2 produces 
higher peaks. Two is a typical value for leniency. 
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Figure 1 19. The effect of leniency on an edge filter image 
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This chapter describes the Cognex Point Registration tool. It contains seven sections. 

The first two sections, this one and Seme Useful Definitions, provide an overview of the 
chapter and define some terms that you will encounter as you read. 

Point Registration Tool Overview provides information about me capabilities and intended 
use or the Point Registration tool. 

Haw ths Point Registration Toot Works provides a general description of the operation of 
the Point Registration tool. 

Comparing the Point Registration Tool With Search compares the Point Registration tool 
with the Cognex Search tool. 

Using the Point Registration Toot describes some of the techniques that you will use to 
tmptemont an application using the Point Registration toot. 

Point Registration Tool Data Structures and Functions provides a detailed description of the 
data structures and functions that you will use to implement your application. 
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Some Useful Definitions 

point registration A searcn technique designed to determine the exact point at 
which two images of the same scene are precisely alloned. 

global minimum The lowest value in a function or signal. 

local minima A low value in a function or signal. 




Local minima Global minimum 



subpixel accuracy Positions within an image may be specified in terms of whole 
pixel positions, in which case the position refers to the upper- 
left corner of the pixel, or in terms of fractional pixel positions, in 
which case the position may lie anywhere within a pixel. 
Positions specified in terms of fractional pixel positions are 
referred to as having subpixel accuracy. 
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Point Registration Tool Overview 

The Cognex Point Registration tool performs fast, accurate point registration using two 
images that you supply. 

Point Registration 

Point registration is the process of determining the precise offset between two images of 
the seme scene. 

You ueg the Point Registration tool to perform point registration by providing a model image 
and a target image, along with a starting point within the target image. The model image 
snoutd oe a reference image of the feature you are attempting to align. The target image 
should be larger than the model image, and it should contain an instance of the model 
Image wttnin it 

The Point Registration tool will determine the precise location of the model image within the 
target image. The Point Registration tool will return the location of this matcn. called the 
registration point with subptxai accuracy. 

Figure 160 illustrates an example of how the Point Registration tool performs point 
registration. 




Figure 160. Point registration 
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The Point Registration toot is optimized to locate precisely the model image within the target 
image, even if the target image contains image defects such as reflections, or if the modei 
image is partially oDscured by otner features in the target image. Fioure 161 illustrates an 
example of point registration wnere the target image is partially obscured. 






Model image 



Target image 



Location of model image 
within target image 



Figure 161. Point registration with partially obscured target image 
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How the Point Registration Tool Works 

The Point Registration tool finds the location of the model image within the target image 
You operate the Point Registration tool by supplying me model end target images along 
with the location within the target image where you expect the origin of the model image to 
be. The Point Registration tool will determine, with suboixel accuracy where the onain of 
the model image lies within the target Image. 

The Point Registration tool works by computing a score indicating the degree of similarity 
between the model image and a particular portion of the target Image that is the same size 
aa the model image. This score can range from o to 253, with a score or 0 indicating that 
the model image and the target image are perfectly similar and a score of 255 indicating 
that the model image and the target image are perfectly dissimilar. The tool computes mis 
score for locations in the immediate neighborhood surrounding the starting point. The tool 
will find the location within thig noignborhood of me target image that produces tne local 
minima in the value of this score. 

By adding an interpolation step, the Point Registration tool then determines the location of 
the model image within the target image with subpixel accuracy. For typical images, the 
Point Registration tool can achieve accurate registration to within 0.25 pixels. 

Because of the way the Point Registration tool aeeka the local minima, if the starting point 

you specify is more than a few pixels from the actual registration point, tne tool may not 
return the correct registration point. The exaot amount of varionoe that the Point 
Registration tool can tolerate will vary depending on the images. The variance may be as 
small as 3 to 5 pixels for some images or as large as 30 pixels for others. 

Point Registration Score 

Each time it la invokea. the Point Registration tool will return, in addition to tne location of 
the origin of the model image within the target image, a score indicating me degree of 
similarity between the modal image and the target image. The score returned by the tool 
will be from 0 to 255, with a score of 0 indicating that the model image and the target image 
are perfectly similar and a score of 255 indicating that the model image and the target 
image are perfectly dissimilar. 

If your point registration receives a high score, the actual precision of the point registration 
location may be somewhat lower than if the point registration receives a low score. 

Exhaustive Point Registration 

When you use the Point Registration tool, you supply the tool with two images and a starting 
location within the model image. The tool will confine its point registration search to a small 
area around the starting location mat you specify. 
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The Point Registration tool can also perform point registration exhaustively, that is, by 
computing the score for every possible location of the model image within the target image. 
This procedure, called exhaustive point registration, is extremely stow. It can be heiDful, 
however, in debugging applications where the Point Registration tool does not appear to 
be working correctly. 

Masked Point Registration 

You can limit the areas of the model image and target image that the Point Registration tool 
uses to perform the point registration by supplying a series of rectangles to the tool. If you 
supply these rectangles, the tool will compute the score based only on those pixels 
contained within the rectangles that you specify. 

Figure 162 illustrates an example of specifying a masking rectangle. In mis example, tne 
right side of the model is often obscured in the target image. By specifying a rectangle that 
oovere the left aide of the model image, you can cause the Point Registration tool to 
consider only the pixels in that part of the model image. This will tend to increase the 
accuracy of the point registration. 



Masking 
rectangle 




Location of model image 
Model image Target image tafget jmage 



Figure 162. Masked point registration 

You can al30 specify several masking rectangles. Figure 163 illustrates an example where 
the oenter of the modal image ta often obscured in the target image. By specifying a series 
of rectangles, you can eliminate from consideration tne part of the image that is most 
frequently obscured. 
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Masking ^ 
rectangles 




Model image 



Target image 



Location of model image 
within taroet imaoe 



Figure 163. Masked point registration uetng multiple rectangle* 

If you specify mulDple masking rectangles and the rectanoles overlap, any Dixels that are 
contained in more than one masking rectangle will be counted toward the score multiple 
times, once for each rectangle in which they are contained. 

In all cases, the Point Registration too! will return the location of the origin of the model 
image within the target image. 



The Point Registration tool may not work well in cases where the model image and the 
target image have different intensity values. You can peiiorm limited image processing as 
part of the point registration by supplying a pixel mapping table. 

It you eupply a pixel mapping tablo. ©very pixel in the target image will be mapped to the 
value in the pixel mapping table at tns location within the pixel mapping table given by the 
Dixei's value. For example, tf a olxei in the target image had a value of 200, it would be 
mapped to the value of the 200*" element within the pixel mapping table. 

IT you nave specified a masking rectangle, only those portions of the target image that lie 
within the masking rectangle or rectangles will be mapped through the pixel mapping table. 

For more information on pixel mapping, see Chapter 3, Pixel Mapping, in the image 
Processing manual . 



Image Normalization 
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Comparing the Point Registration Tool with Search 

This section compares the Point Registration tool with me Search tool. For more detailed 
information on the Search tool see Chapter h Searching. 

The Point Registration tool has the following advantages over the Search tool: 

• It is less sensitive to images that contain areas of pixels that have widely different 
values than the same pixels in the model image (i.e.. blotches or occlusions). 

• It requires no training step. 

The Point Registration tool has the following disadvantages when compared with the 
Search tool: 

• It is capable of fine alignment only. 

• it does not work with scaled or rotated target images. 

• It does not work weli with images with brigntness changes. 

• It will not find or rank multiple registration points. 
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Using the Point Registration Tool 

This section describes how to use the Point Registration tool. 

Obtaining Model and Target images 

You acquire the images that you use as model and target images uang the techniques 
described in Chapter 7, 3000/4000 Acquisition and Display, in the 3000/4000 Image 
Acquisition and I/O manual and Chapter 1, 5000 Acquisition and Display, in the 5000 
image Acquisition ana I/O manual. 

You should acquire all images mat you plan to use with the Point Regisvation toot, 
especially images of scenes with low contrast, using a bit depth of at least 8. For fastest 
performance, you should acquire images as FAST8 images. 

When you acquire the model image, you should take care to ensure that the image is as 
ideal a model as possible of the images that you will be using as target images. The Point 
Registration tool is designed to align target images wlm a wide variety of image defects, 
but in order for the tool to work, the modol imago need9 to be as fro© from defects as 
possible. 



Specifying a Starting Offset 

When using the Point Registration tool, you must specify a starting location in the model 
image. The tool will perform its point registration search starting at the point that you 
specify. Because the tool only seeks the local minima tor the score value, if you specify a 
starting location that vane© greatly from the actual registration point, the point registration 
operation will fail. 

You can use other vision tools such as the Search tool or the Inspection tool to locate the 
model feature within the target image, or you can rely on operator input to specify the 
coarse location of the feature. 
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Point Registration Tool Data Structures and 
Functions 

TNs aeciion contains descriptions of the Point Registration tool data structures and 
functions. All Point Registration tool data structure and function names begin with the Drefix 

Structure creg_params 

cr«g_panifra contains parameters that determine now the point registration searcn will be 
conducted. A structure of this type is supplied as an argument to creg_po*m_regl«terO 

Iinclu.de <register.h> 

cypadef struct 
( 

c_Int32 start_x. 

c_ Inc32 rect_count; 
char 'optionaljnapf 
c_Int32 flags; 
) crea-P&rams ; 

• starts and $tan_y are tne x- and y-coordinates of the starting position for the point 
registration searcn. 

• racLcounris the number of rectangles in the rects argument to crecpolnt-regiaterO- 

• optionaLmap is an optional pixel map. If opbonaljnap is non-NULL, the Point 
Registration tool will replace each pixel in the t ergot image with the value contained in 
the element of opuonaljnap corresponding to the value of the pixel in the target 
image. If you cup ply a value for optionsLmsp. you must eupply en image of type 
FAST8. 

You can use optionat_map to compensate lor image-to-image brightness variations. 

• (tags specifies tne type of point registration inaT tne Point Registration tool will perform. 
flags is constructed by ORing together any of the following values: 

• CREGJTYPE1 is reserved for future use; you should always include CREGJYPE1 
in tne value of flags. 

• CREGJTYPE2\s reserved for future use; you should never include CREGJTYPE2 
In the value of flags. 

• CREG_NORMAL is useo to specify a ooint registration based on finding tne local 
minima of the score value. 
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• CREG.EXHAUSTIVE is used to specify a point registration based on finding me 
global minimum or trie score value. 

If you specify CREG_NORMAL and if s/arr_xand starty are more than a few pixels 
from the actual registration point, the tool may return a location that does not represent 
tne Dest matcn witnin tne Image, n you speciry CREG^EXHA USTIVE, The registration 
will return the be9t registration matcn tor the entire image. An exhaustive point 
registration will be extremely slow. 

Structure creg_results 

creo_pomt_j-»gl«ter returns a pointer to a crag_retutt« structure. crag_po|r*_regl«Ur 

fills in the structure with information about the results of a point registration operation. 

•include <register.h> 

typedef struct 

( 

float x, 

y; 

c_Int32 ecore, 

time; 
chAt on_edge_x, 

on_edge_y ; 
) creg.re suits; 

• x and y are the x-coordinate and y-coordinate. respectively, of the location within the 
model image at which the target image was tound. xand y give tne position with 
subpixel accuracy. 

• score is a measure of the similarity between the pixel values in tne model image and 
tne target image at tne nearest whole pixel alignment position, score will be between 
0 and 255. with lower values indicating a greater degree of similarity between the 
model image ana tne target image. 

• time is the total amount of time that the tool spent on this point registration, in 
milliseconds. 

• on_edQ9_x is sot to a nonaero value if. along the x-axic, one edge of the model image 
area is at the edge of the target image. If on^edge^c is nonzero, the accuracy of the 
position information may be slightly reduced from what it would otherwise be. 

• or^edge^y is set to a nonzero value if. along the y-axis, one edge of the model image 
area is at the edge of the target image. If on_edge_y Is nonzero, tne accuracy of the 
position information may be slightly reduced from what it would otherwise be. 
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Function creg_point_register() 

cr«g_palnt_regf«t*r() performs point regiotroDon ucing the model imago and target image 
that you supply. The point registration will be controlled by the parameters in the 
crea-Pwns structure suDDtied to the tuncuon. crog_j>olrit_reglatar() rfltumB fl pointer to 
a creg.resutta structure that describes the result of the point registration. 

finclude <register.h> 

creg^results *creg_point_register ( conrc cip_bu£fer 'target, 
const cip_£uffer *model, const cia_rect *rects, 
const creg_paraiti6 'parajna, creg_resulcs *ro£UltB) ; 

• target points to the image to use as the target rmag© for thic point registration, target 
muss be at least one pixel larger in both the x-dimansion and tne y-dimension than 
model. 

• model points to the image to use as the model image for this point registration, model 
must be at least one pixel smaller in both the x-dimension and the y-dimension than 
target 

• rects points to an array of cla.r act structures. If rects is non-NULL the Point 
Registration tool will limit the comparison of pixel values to just those pixels that lie 
within me reaangtes containea in rects. If the rectangles in rects overlap, a pixel will 
be included in the comparison once for each rectangle That contains it. 

• params points to a creg_params structure that defines the parameters to use for this 
point registration. 

• results points to a creg_re*utt structure. creg_polnt_reglater() will place the results 
of thia point registration in results and will return a pointer to results. 

it you supply NULL lor results. crog_polnt_r«gt«t«r() will allocate a crag.reault 

structure from the neap using the default allocator. You can free this structure by 
calling froe() 

cregj>olnt_regl«tor() throws CGEN_ERR_BADARG it 

• rects is NULL and the recLcount field of params is not 0 

• If model & larger than or the acme size ad target 

• If the starts field In params is less than 0 or greater than the width of target minus the 
width of model 

• If the stany field In oarams is less than 0 or greater than the height of target minus the 
height of model 

• If model is NULL 
If params is NULL 
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• If target is NULL 
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An Overview of the Line Finder 

The Line Finder computes the locations of lines in an image. For each line that it finds, it 
returns an angle B and a signed distance dfrom the central pixel of the image. Given 8 and 
d, you can compute ail points (x.y) in the image that satisfy the equation of the line (as 
illustrated in the section The Transformation from Canesian Space to Hough Space on 
page 204). The algorithm that the Line Finder uses, along with the angle/distance 
classification of lines, is explained in How the Line Finder Works on page 166. 

You can use the bne Finder in either of two modes, quick or normal . In quick mode the Line 
Ftnaer provtaes tne location or me point along eacn nne tnat is nearest to me center or tne 
image, along with the angle of the line. In normal mode, the Line Finder also estimates the 
length and density of each line that it locates, using statistical methods, and returns the 
position of the center of mass of the line. 

Figure 78 illustrates the two modes of the Line Finder. The image in Figure 76a Is snown in 
Figure 78b after the Line Finder has been invoked in quick mode. The presence of the four 
lines is indicated. Each line is drawn to the screen at a constant length, but no attempt ts 
made to estimate its actual length. Figure 78c shows the result when using the Line Finder 
in normal mode: the length of each line segment is estimated. 

For a aetauea explanation or me aaia returned oy tne Line Finder in eacn mode, see tne 
section Results in Quick Mode and Normal Mode. 




a b c 



Figure to. An image (a), tines touna in quick moae (D), ana nne 
segments found in normal mode (c) 

The Line Finder finds tines regardless of grey scale variations In the image, rotation of the 
scene, or changes in scale. A typical application for the Une Finder is locating fiducial 
marks in a series of images in which grey levels, scale, and rotation are unpredictable from 
one image to the next. Figure 79a and Figure 79b illustrate two images, each containing 
the seme fiducial mark: a square within a square. However, the sizes of the marks, and their 
rotations differ netween the two images. Also, the mark in Figure 79a is a dark square within 
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a lighter square against an even lighter background, but the mark in Figure 79b is a light 
square within a darker square against a darker background. As shown in Figure 79c and 
Roure 79d. the Line Finder finds the outlines of the fiducial marks aesDite these variations. 




Figure 79. The Line Finder is immune to grey scale, size and rotation 
changes 

Since me Line Finder Is an intermeaiate-ievei tool, your application wm proDaDly require 
that you do further processing, using the results of the Line Finder as input. You will 
probably not be searching lor linos in your image, out rather for a feature wnose presence 
is implied by the locations of lines. For example, in Figure 79, the Line Finder does not 
return the locations and dimensions of the fiducial marke, only the locations of the lines that 
bound the nested squares. Computing the precise locations and sizes of the fiducial marks 
requires post-processing based on the Line Finder's results. 

The Line Finder demo code includes post-processing of this sort. It contains functions that 
locate squares of any size and rotation, using the Line Finder. This code deduces the 
presence of squares based on the Line Finder's results, eliminating extraneous lines. See 
Chapter 1, CognexAPI Introduction, in the Development Environment 1 manual tor the 
location and name of the Contour Finding demo code on your system. 
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How the Line Finder Works 



This chapter explains how the Line Finder finds lines in an image. This discussion is divided 
into the following cub cacti one: 

• Defining a Line by Angle and Distance describes the definition of a line used by the 
Line Finder. 

• Hough Space introduces the two-dimensional space in wnich the Line Finder records 



• The Hough Line Transform describes the algorithm that the Line Finder uses to find 
lines In an Image. 

The Line Finder uses a Cartesian coordinate system whose origin is the center of the image. 
Most of the examples in the book are drawn in this coordinate system, although a few use 
image coordinates. These two systems are pictured in Figure BO. Notice that the x-axes of 
the two systems are the same, but the y-axes are opposite. Note also that positive angles 
in Cartesian coordinates are measured counterclockwise from the x-axis; positive angles 
In image buffer coordinates are measured clockwise from the x-axls. 



Figure 80. Image buffer coordinates and Cartesian coordinates 

The Line Finder uses the central pixel of the image as a reference point. The image buffer 
coordinates of this pixel (x c% yj are computed with integer division, using the following 
formulas: 



tines. 



-y 
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y c = H/2 

where wis me wiatn or me image ana Mis rne neight of the image, in pixels, in the Line 
Finder, the Cartesian coordinates of the central point in the image are (0 t 0). 

Defining a Line by Angle and Distance 

The Line Finder defines a line (in Cartesian coordinates) by its angle from the x-axis and by 
the signed distance from the center of the image (the central pixel) to the line. These two 
parameters uniquely describe any line in Cartesian space. 

The feature in Figure 81 a contains an edge that lies along a line (Figure 81b). The line is 
defined by its angle from the horizontal (0) and by the shortest distance from the center of 
the image to tne line (Figure 61 c). Note that the distance vector is perpendicular to the line. 




a b c 



Figure Angle ana distance aefine a tme 

In the definition of a line by angle and distance, the distance is negative if the angle of the 
distance vector (in Cartesian coordinates) is greater than or equal to 0° and less than 180°: 
the distance is positive if the distance vector has an angle greater than or equal to 1 80° and 
less than 360°. 

Distance is signed to avoid the ambiguity illustrated in Figure 82. Figure 82a shows an 
image wttn two features. Eacn feature is bordered on one sloe oy a line. Aimougn me lines 
are different, they have the same angle e and the same absolute distance dfrom the center 
(Figure 62b). 
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c d 



Figure 82. The angle of the distance vector determines the sign of the 
distance 

By designating a sign to the distance vector, based on its angle (6 in Figure 82c, 5+180 in 
Figure 82d). each line is ensured a unique definition. In this example, the two lines are 
defined as (6, d) and (e, -d). 

Once you Know the distance and the angle of a line, you can compute all points fx* y 0 \ in 
Cartesian coordinates, that lie on the line. Given a distance, dand an angle e, a line is ell 
points (x* y e ) that satisfy the following equation: 



d = x 0 sin6 - y 0 coi8 

The derivation of thie formula ie supplied in tho Motion 7ho Transformation from Cartesian 
Space to Hough Space on page 204. As an example of its application, if 8 is 45 and dis 0, 
a line is all points tn Cartesian space that satisfy the following equalities: 
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0 = xsin(45) - yco$(45) 



0 - x-L -y 



t 



x = y 



Hough Space 

Hough space is a two-dimensional space in which the Line Finder records the lines thai it 
finds in an image. A point in Hough space represents a line; one axis represents angle, the 
other represents distance. 

Hough space is implemented in the Line Finder as an array: a simple example is shown in 
Figure 83. The Hough space in this example can record eight angles and nine distances 
and therefore 72 lines. 

The Line Finder lets you conrrol the size of Hough space. You specify the range of distance 
in pixels, and, in the edge detection data structure thai you pass to the Line Finder, you 
supply the number of angles that the Line Finder can compute. 



315 

270 

225 

Angle 180 

135 

90 

45 

0 1 | I [ I I | I ] 
-4-3-2-101234 
Distance 

Figure 83. Hough space 

The Hough space in Figure 84 contains a single line. It has an angle of 135° ana is a 
distance of 3 units from the center of the image. 



79 



WO 98/18117 



PCT/US97/18268 



3 Line Finder 



Angle 



315 
270 
225 
1B0 
135 
90 
45 




-4-3-2-101234 
Distance 



Imago 



Figure 84. Hough space containing a single line 



The Hough Line Transform 



The Line Finder uses the Hough Line Transform to locate lines in an image. This method is 
outlined and illustrated in this section. To understand this discussion, you should be familiar 
with edge detection and peak detection, as described in Chapter 4, Edge Detection Tool, 
in the Image Processing manual . 

The Hough Line Transform, as implemented t>y this tool, takes advantage of the following 
rule; If an eage pixel 's angle 9 ana location (x0,yO) 15 Known, men tne line on whicn the pixel 
lies is also known: it is tne line with angle ^90 that contains the oixel (xO,yO) This is 
illustrated in Figure 86. 
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271 



i 



270° 



b 



c 



Figure 85 Four eages in an image (a), eage angles at four eage pixels 
(b). ana line angles at those pixels (c) 



1. 

2. 
3. 

4. 
5. 



Hough Line Transform is implemented as follows: 

Using edge detection, the Une Finder creates an edge magnitude image and an edge 
angle image from the input image. 

The Line Finder creates and clears a Hough space. 

For each edge pixel m the input image, the Line Finder does the following: 

♦ Using the edge angle * and location (xO.yO) ot the pixel, the Line Finder calculates 
the line's distance from the center. d t and its angle, e (using the formula e = o -r 



♦ The Line Finder increments the bin in Hough SDace representing the line (0,d). 

Once all edge pixels in the image have been examined, the Line Finder searches 
Hougn space for maximum values, using peak detection. The highest value in Hougn 
apace represents the strongest line m the image, me second highest represents the 
second strongest Hne. ana so form. 

The location of each line is printed out and, optionally, each line is drawn in an output 
image. 

Figure 86 through Figure 69 illustrate this method. 

Figure 66a ts an image passed to the Une Finder. Figure aeb is a stylized edge 
magnitude image created from the input image. 
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Figure 66. An input image (a) and (he edge detected input image (b) 

Each edge pixel is processed, as described above. For example, because the outlined 
edge pixel in Figure 87 belongs to (he line (90, -2). the bin in Hough space tnat represents 
that line is incremented. 
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Figure 87. The bin in Hough space for the edge ptxei's line is 
incremented. 
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Figure Sfl shows tne final Hough space along with the input image. Figure 89 shows the 
peak-detected Hough space, with each of tne peaks outlined. Figure 89 also shows the 
input Image with the iines, represented by the peaks in Hough space, drawn into the input 
image. 
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Figure 88. The final Hougn space and the input image 



83 



WO 98/18117 



PCT/US97/18268 



3 Line Finder 



Angle 



315 
270 
225 
180 
135 
90 
45 
0 



0 


4 


0 


0 


0 


2 


0 


0 


0 


0 


1 


0 


0 


0 


2 


2 


3 


0 


0 


0 


2 


0 


2 


1 


0 


0 


0 


0 


0 


0 


1 


0 


0 


2 


1 


0 


0 


0 


0 


0 


0 


0 


0 


2 


0 


0 


0 


7 


0 


0 


0 


0 


1 


0 


0 


1 


2 


0 


0 


0 


0 


i 


0 


0 


0 


1 


1 


0 


0 


0 


0 


0 




Distance 

Figure 89. The peafrdetoctad Hough space ana tho input image, with 
lines added 

Notice that the Hough space contains many extraneous lines. If, for instance, tne vtsion 
problem is to find the triangle in the image, the lower scoring lines that border the blob need 
to be eliminated. Almost all of the Line Finder applications require post-processing of this 
sort In this example, simply locating the three highest peaks in Hough space eliminates 
the unwanted lines (Figure 90). In most applications, the required postprocessing is more 
complex. 
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Figure 90. The output image after postprocessing 
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APPENDIX II 



Patent Application for 



MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION 



Software Listings 



THE FOLLOWING APPENDIX IS NOT BELIEVED TO BE NECESSARY FOR 
ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED AND 
CLAIMED IN THE ACCOMPANYING APPLICATION. 
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* Copyright (c) 1996 by Cognex Corporation, Natick. MA USA 

* All rights reserved. This material contains unpublished. 

* copyrighted work, which includes confidential and prcpneta: 

* information of Cognex. 



* cwa_f ind_target . c, Aaron Wallack - 13 Apr 1996: Created. 

* Time-stamp: <96/07/30 09:32:47 awaliack> : last modified. 



SRevision: /main/noneS 



#inciude <prealign. h> 
^include <math.h> 
^include <stdio . h> 
^include <cct.h> 
#include <cip.h> 
^include <ctm.h> 
^include <caq/caq_def . h> 
# include < cwa_cr i v . h > 
^include <search/ctr def.h> 



#include 
^include 
^include 



<search,'cse def.h> 



:lp/ ccip_er . h> 



# include <chp . h> 



/* This file includes a 
* square) target . 



.oca-icir.c: a crosshair 



* The algorithm is intended to handle the cases when the target is 

* imperfect (occluded or broken) 

* (occlusion) 

TTftnftwwtTrfrtriftr 
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broken 

############### 

These routines are intended to be used as a subroutine for 
determining the cameras' field of views (with respect to an absolute 
coordinate system defined by a motion stage) . 

The target localization routine (cwa_f ind_target) takes as input: 

img - a cip_buffer containing an image of the target 

calib - a calibration structure intended to characterize the 

non-rectangularity of the pixels 
angle - the orientation of the crosshair in terns of image 

coordinates (we require this 

orientation estimate tc be accurate :: 5 degrees, : . ? 5 radians 
result - pointer to a cva_Iir.e structure where the result will be 
returned 

Requirements : 

Image quality ; 

The image of the crosshair must be visible and perfect 
(without occlusion or breaks"; throughout a IjG X IOC pixel square 
surrounding the crosshair center. 

Consequently, the center of the crosshair must be ao least 
50 pixels away from the borders z f the cip_cuf fer . 

Orientation estimate accuracy: 

The given angle estimate (given ir. terms cf image coordinates ^ must be 
accurate to within 3 degrees 0.25 radians 

This algoritm tolerates occlusion and breaks in the target 
so long as these imperfections occur at least 50 
pixels away from the crosshair center 

Major Consideration: 

One interesting thing to note about crosshair calibration, and the reason 
that we use four distinct rays, is that the separation between the opposite 
rays depends upon the lighting and aperture of the lens. Furthermore, we 
cannot assume that the separation between these edges will remain constant 
from one setup to another 

What we really want is the position of the ■ *' 

4 TT ######## ttw 
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############### 



Since we do not know what the of f set / separation is, it turns out that the 
repeatability of the entire fit (localization of the crosshair) is bounded 
by the accuracy of the smallest of these four lines. It doesn't make any 
sense to include 10000 edge points from one ray if we only have 50 edge 
points from another (the localization error will still be on the f- d «^£ 
0 . l/sqrt (50) pels, assuming random i.i.d. error and suc?:;:e. .cca-izat icn 
accuracy of 0 . 1 pixel). 

we want to incorporate the same number of edge points from opposite rays 
because we always want to fit the lines to the same exact points oor 
highest accuracv and repeatability' - When the oalioration target is 
occluded by a particular object (such as the borders that can be seen m 
the database) . then if we use the exact same number of pixels on tne 
opposite unoccluded ray as the number of pixels on the occluded ray, tnen 
we will always be locking at the same pixels on the unccciuded ray 

Basically, since we dc not know which edge points correspcnc to 

L - . ~ -so. - -a - — r» «- -a ~ ^ * o^c:^ oomts corresoond to 
imoerr ect ions m tne r e _ a— v* — _- r _ -> ~ - 

valid points on the target, we use a beet - strapping a.goritnm to 
compute regions where edge points are • 

Assuming that the target is perfect within the GUARANTEE GOOD 
REGION (center + /- 50 pixels), we can compute four GCCD RAYS 
characterizing the four rays emmatmg from the crosshair center. 
Then, we can use these GOOD RAYS to define the VALID REGIONS (the 
set of points within a TIGHT_THRESKOUD of the GCCD RAYS). Finally, 
we improve the estimates of the four rays by going back and 
enumerating edge points along the GCCD RAYS until we encounter an 
> edge point which falls outside the 7ALII: REGIONS. 

* Definitions 
* 

* GUARANTEED GOOD REGION 

* region surrounding the origin where we insist that the calibration target 

* must be perfect 

* GOOD RAY 

* A ray fit to points enumerated w-hm the guaranteed ?coc region 

* VALID REGION 



89 



WO 98/18117 



PCT/US97/18268 



Region surounding good ray where we trust chat the edge points are due to 
Che crosshair and not from occlusion or breaks 

OUTLIER POINTS 

Points which we cannot trust because they do not fall within the valid 
region 

CENTER OF CROSSHAIR TARGET 

V outlier 
v edge points 

< good ray > 

VALID REGION 

t* tt # tr -r «r tr n 

•< guaranteed gccd regicr.- •• 

We are adopting a better- safe - than- scrry strategy ftr handling 
occlusion (as seen as we see a "bad" pemt. we lese faith m ail of 
the subsequent pomes) . Although this approach is subcorneal, the 
database images seem tc contain nearly perfect points emir.acmg 
from the origin that it should work well in practice. 

Algorithm: 

1; Use absolute normalized ccrrelaticn search to approximately 
locate the crosshair center 

This involves constructing a synthetic rr.cdei cf a rtcaced 
crosshair center isee grcss_lccate_crcsshair_cencer_m_i:riage . '. . 

2) Given this initial positicn estimate and the user-supplied orientation 
estimate, improve the positional and crientational estimates by localizing 
four good rays eminating from the crosshair center. This mvoives 
enumerating edge points within the GUARANTEED GOOD REGION (50 pixels) 
along the supplied orientation as well as 

theta+PI/2, theta+PI, theta+3*?I/2 . These computed four GOOD RAYS are used 
in step 3 to check whether edge points are outliers. In other words, these 
four good rays define VALID REGIONS separating good edge points (points we 
believe correspond to the crosshair) from bad edge points (points we 
believe are due to occlusion or breaks) We should also point out that we 
fix the orientations of the four rays by "averaging" these orientations, 
i.e., we incorporate the constraint that the four rays correspond to 
orthogonal physical lines. (see 

compute_center_position_and_angle_assummg_f our_guaranteed_good_rays ( ) ) 

3) Go back and enmerate edge points along the four rays until we find an 
edge pcint which dees not satisfy the VALID REGION constraints. We also 
throw out some of the edge points in order tc realize equal numbers of 
edge points from opposite rays. (see 
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compuce_center - positicn_and_angle_aicng_guarar.teed_gzcc_rays ; ; 
NOTES 



When we enumerace edge points along rays eminating from the origin, 
we scar: out a small distance away from the origin sc that we don't 
gee confused by two different edges (this buffer zone is 
characterized by the constant C:RCUIJVR_BUFrERjZCOT:_ARCUND_CZ^rrE?. . 
which is 10 pixels) 

The central function in this algorithm is 

enumerate_points_and_f it_line_along_ray { ) which enumerates edge points 

* along a specified path. enumerate_points_and_f it_line_aiong_ray * ) applies 

* calipers at sample points along the path (the edge estimates car. be 

* further improved using parabolic subpixei interpolation; Along a specific 

* path, enumerate_points_and_f it_ime_along_ray { ) always makes integral unit 

* steps m either the x or y direction. This is sc that we can apply 

* axis -aligned calipers along every row or column when the step size is 1 

* enumerate_pcir.zs_and_f i t_lme_alzng_ray * ; enumerates edge points 

* until it either encounters a bad edge pcir.z, zr finishes sampling 

* the edge segment. At this point, it ceases enumerating edge points. 

* and fits a line to the points. There are fzur ways an edge pemt 

* can be "bad" . If enumerate _j:cmts_and_f it_lme_alzng_ray ■. 

* encounters any of these four situations, it returns Z after fitting 

* the line; if it successfully enumerated all z: the edge points 

* along its specified course, Lz returns 1 . 

1. no edge is found by the caliper applied at the sample point 

2. more than one edge is found by the caliper and the highest 
scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the CONFUSION jrHRESHC 1-2 ; 

3. the computed edge point is an outlier '.the distance between the 
edge point and the nominal line is larger than threshold) 

4. the caliper extends outside of the cip_buffer 

* When we call enumerate_pomts_and_f iz_line_aiong_ray ( ! in step 1, we 

* specify a ray from a point near the crosshair center to a point 50 

* pixels away from the crosshair center. We do look at the return val 

* to check that the enumerate _point3_ar.d_f it_lme_aiong_ray { ) did not 

* encounter any problems . 

* When we call enumerate_points_and_f it_lme_along_ray i) in step 2, we 

* specify a ray from a point close to the crosshair center along a 

* good ray to a point 1000 pixels away (guaranteed to be outside the 

* cip buffer) . Thereby, even if we do not see any "bad" points, we 

* will undoubtedly leave the zip_buffer at some point. 

* The reason we pass a calibration object to cwa_f ind_target is that we 

* recompute the orientations of the four rays by incorporating the knowledge 
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that they all come from orthogonal lines . If the pixels are ncn- square 
(i.e., by virtue of -he face chat the CCD elements are r.cn - square ; , then 
we need to account for this when we "average" the f cur orientations . We 
"average" the four orientations by transforming the image orientations 
into physical coordinates (and then truly averaging tr.e physical angles; 
and then transforming the average angle back into image coordinates . We 
try to use the term "phys" to denote physical coordinates and M img M to 
denote image coordinates (such as phys_orient and img_crient J . 

We use different sized calipers for steps 2 i 3 because we make 
different assumptions about the relative accuracies of our caliper 
application points. We use the terms LOOSE and TIGHT to refer to the 
calipers; LOOSE for step 2 where we have little faith m the 
application points, and are thereby forced to use broad calipers, and 
TIGHT for step 3, where we have much mere faith in the application 
points, and can thereby use relatively tight calipers. 

We expect the initial search position to be accurate tc 2 pels, and 
the orientation estimate to be accurate zz 0 . 35 radians. Ac a caliper 
application point 50 pels away, the caliper may be eff cy up :: S 
pels from the correct edge positicn. Just tc be safe, we use a 
caliper with projection length 2 4 !12 cn each sice: tc enumerate edge 
points in step 2 

After improving the positicn and criencaticn estimates m step 2. we 
basically assume that we've get the right positicn tc - - 0.2 pels 
and +/- 0.01 radians. Thereby, we can use these lines threshold 
outliers because we would expect errors of only 2.2 pels at 20C 
pixels away. This is pretty tight because we use a threshold of 3.0 
pels to detemme outliers 

(actually, the 0.2 pels 
performance because the 
end up predominantly af 
final line estimates: 

Main subroutines - 

gross_locate_crcsshair_center_ir._image ; ) 

construct a synthetic model of the crosshair and use search to 
estimate the crosshair position 

f it_line_to_pomts [) 

fit a line (x,y, t) to an array of points 
compute_subpixel_position ( ) 

use parabolic interpolation at the neighborhood of the application point 
to compute the subpixel edge position 

enumerate ooints_and_f it_iine_along_ray ( ) enumerate edge points along a 
line segment from start — point to endjeoint and possibly store the fcund 
edge points in an array. Furthermore, a nominal line and a threshold are 
passed in as arguments , and when the found edge points deviate from this 
nominal line by mere than the threshold, enumeration is stopped and a line 



. 0.0* radians couic s 
first 50 pels and the 
fectmg which edge poi 



ignificantly impact the 

orientaticn estimate 
nts are included m the 
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* is fin co the enumerated edge positions. Enumeration is also stopped when 

* two edges are found (confusion) , no edges are found {occlusion; or when 

* the caliper is applied out of hounds (out of bounds; 

* average_f our_image_orientations ( ) 

* average the orientations of the four rays to compute the best 

* composite orientation estimate. This function transforms the image 

* oriencations into the physical domain in order to expicit the 

* constraint that the physical lines are orthogonal 

* compute_center_position_and_angle_assuming_f our_guaranteed_good rays () 

* improve estimate of the crosshair position and orientation'by 

* searching four regions of guaranteed good length (50 pixels) 

* eminating from the crosshair center. Lines are fit to the edge 

* points along the four regions, and then the orientations of these 

* lines are "averaged". Two orrhogonal lines are computed by 

* averaging pairs of the four rays Finally, the position estimate is 

* computed by intersecting the two orthogonal lines 

* corr.pute_center_pcsition_and_angie_aicng_guaranteec_gccd_rays ; , generate 

* final estimate of crosshair position and crientaticr. by enumerating :c:r.:3 

* along four rays (using four good rays as thresholding lines . We use 

* tighter calipers (than we used in step 2. because we trust the positions 

* of the good ravs . 
V 



^define N_EDGES 10 

^define GUARANTEED_GCCD_SIZE 50 

^define GROSS _TARGET_S I ZE GUARANTZZD_GCOD_S I ZE 
#def me NUM_S AM PLZ_ POINTS 1000 
^define LOOS E_THRES HOLD 12. 
^define TIGHTJTHRESHCLZ 3. 
itdefine LOOSE_CA^I?ER_WIZTH 24 

/* ( { mt) 2* \LOCSE_THRESHCLD*G*JAilA;rrEED_GCCD_s:ZE*sin iO . :5, , ) - 2 (kernel)* 

^define TIGHT_CALrPER - WID7H 14 

/* ( i mt ) 3 *TIGKT_TKRESHGLD) - 2 (kernel;*, 

/* 

* TIGKT_CALI?ER_WIDTH should be larger than the minimum width so that 

* we check whether chere is a sharp edge just outside the 

* T I GHT_TKRESHO LD rang*, because if there is a sharp edge nust 

* oucside the TIGHT_THRESHOLD range, then we want to declare 

* confusion and stop enumerating ooints 
*/ 



^define LOC S E_CAL Z PER_ PROJECT I ON_ LENGTH 5 

#def me TIGHT_CALIPER_PROJECTION_LENGTH 5 

fcdefine MAX_NUM_ POINTS 1000 

ttdefine EPSILONJTHRESHOLD 0.05 

^define CIRCULAR_3UFFER_20NE_AROUND_CENTER (8. + LOO S E_CAL 2 ? ER_ PROJECT Z CN_ LENGTH) 

^define STE?_SIZE_USEDJTO_ENUMERATE__POINTS_IN_GOOD_R£GICN 3 

^define STEP_S I ZE_USED_TC_ENUMERATE_ALL_ POINTS 1 

#define MAX_LENGTH_DIAGONAL_ACROSS_CIP_BUFFER 1000. 

^define CONFUSION THRESHOLD 3 . 
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/* number cf pixels to ignore when we run :r.:: an c; 

^define OUTLIER_BUFFER 7 

extern inc cd_showas ( int . char*); 

extern double cwa_point_distance ( ) ; 

char* cy_jprealign_cwa_f ind_ target f ) { return rILE_ 

static int cwa_f ind_target_debug_f lag=0 ; 
int set_cwa_f ind_target_debug ( int x) 

{ 

int old_f lag=cwa_f ind_target_debug_f lag; 
cwa_f ind_target_debug_f lag=x; 
return old_£lag; 



typedef enum calib_orient 
{ 

CALIB_HORIZ=0, 
CALI3J/ERT 
} cai ib_orienc ; 

/* A training parameters record for training rotated rrcssr.a-.r r.cz e. 
scatic ccr_params cwa_calib_ccrr_tp = 

(2, /* make a binary model .2 grey levels 

0, /* no leniency 

0, /* don't measure angle 
5. /* default bias 

1, /* left tail at 1% 
1, /* right tail at 1\ 
SO, /* threshold, not used - 
4,4, /* model resolution 
0, /* don't train for reader 
0, /* no angle training 
0, /* ditto 

0.0, /* no standard deviation thresholding for statistics 
12 8, /* care threshold for mask training 
0 /* NO flags set 

/* (do net bother to initialized reserved fields; 

}; 

static cse_params 

cwa_calib_sp = {cse_absolute , cse_abs_binary , NO, NC, 1, 150, 1000} ; 

/* caliper used to find more accurate position estimates of edge 
* positions 
*/ 

static 

cclp_constraint cwa_f ind_target_gcs [1] = 

{ 



(CCL? CONTRAST, 0, 1000, 0, 100, 
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/* scacic */ 

cclp_caliper clp_f ind_target_initial_search = 
LCOSE_CALIPER_WIDTH, /* Search Length */ 
LOOSE CALIPER PROJECTION LENGTH./* Projection Leng: 



2, 
0, 

EDGEJTHRESHOLD, 
CCL?_DONT_CARE . 
CCLP_NO_PAIRS , 
CCL?_DEFAULT , 
8, 

TRUE, /« 
TRUE , h 
1, 



/* Edge Filter Size 
/* Edge Filter Leniency 
Expected Size */ 



Contrast threshold 
El Polarity 
E2 Polarity 
Optimization 
Bits per pixel */ 
Window Rotation */ 
Interpolate V 



/* Number of contramts */ 



cwa_f ind_target_gcs , /* Constraints 
NULL, /* Pixel Map 

0, 0, 0 



/* static •/ 

cclp_caliper clp_£ ind_target_f ine_searz 
TIGHT_CALI?ER_WI3TH, /* Search Lengt 
TIGHT CALIPER PROJECTION LENGTH./* 



2, 
0, 

10. , / 
EDGEJTHRESHOLD, 
CCL?_DONT_CARE . 
CCLP_NO_PAIRS , 
CCLPJDEFAULT, 
8 , 

TRUE . / 
TRUE . / 



/* Edge Filter 
/* Edge Filter 
Exoeczed Size * ■ 



?r: 
Si: 
Le: 



- 1 1 
e 

ler.cv 



/* Contrast thr 
/* El Polarity 
/* E2 Polarity 
/* Optimization 
/♦ Bits per pi:< 
Window Rctaticr. 
Interpolate 
1, /* Number of cc 

cva_£ md_zarget_gcs , /'* Constraints 
NULL, /* Pixel Map 

0,0,0 



isncia 



i::a:r.:: 



/* static */ 

cclp_caliper clp_f ind_target_compute_subpixel = 
TIGHT_CALIPER_WIDTH. /* Search Length */ 
1,/* Projection Length */ 



2, 
0, 

10. , / 
EDGE_THRESHOLD . 
CCLP_DONT_CARE , 
CCLP_NO_PAIRS, 
CTL?_DE FAULT , 
8 , 

FALSE. 



/* Edge Filter Size 
/* Edge Filter Leniency 
Expected Size */ 

/* Contrast threshold 

/* El Polarity 

/* E2 Polarity 

/* Optimization 

/* Bits per pixel v 



/* Window Rotation 
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FALSE, /♦ Interpolate 

1 , / * Number of concramcs * 

cwa_f ind_target_gcs, /* Constraints 
NULL, /* Pixel Map 

0, 0.0 

); 

/* make an image of a rotated center of a crosshair, this i-age is 

* used to train a model to search for the crosshair in ar. image 

* img - cip_buffer where rotated crosshair will be stored 

* orientation_in_tenths - orientation of rotated crosshair m tenths 
+ of degrees 

* returns img 
*/ 

cip_buf f er *make_cross_hair_cent er_image 
(cip_buffer *img, int orientation_in_tenths ) 

{ 

cip_buffer win, *xhair=NULL; 
cct_signal sig; 

NO_REGISTEH {xhair: ; 

if ( : img) 

cct_error i CGE:*_ERR_3ATARG :* ; 

if (sig=cct_catch;0) ) 
goto dene; 

/ * allocate a temporary ci?_buffer {xhair which will ccr.tair. a 

* non- rotated crosshairs, and then use this temporary np_huf fer 

* as a source fcr cip_rctace rer.tered 

xhair = cio create ( irr.c- > width* 2, ixc - >heicht*C , 3 ; 
cip_set (xhair , 0] ; 

cip_window (xhair , &wm, 0,0, xhair- > width.' 2 , xhair - > height . 2) ; 
cip_set (&win, 255) ; 

cip_window (xhair, iwin, xhair - > width/ 2 . xhair - >height/2 . 

xhair- >width/2, xhair- >height/2 J ; 
cip_set (&win, 255) ; 

cip_rotate_centered (xhair , img , orientaticn_in_tanths; ; 
done : 

if (xhair) cip_delete (xhair } , xhair=NULL; 
if (sig) cct_throw(sig) ; 

return img; 

} 

/* construct a synthetic image and use search to fir.d a grcss estimate 

* of the center position of the crosshair center 

* we expect that the synthetic model will be accurate tc . 
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* approximately 1 pixel, and therefore, this search :cu::r.e snculd 

* localize the center to approximately 1 pixel 

* img - image containing a rocated crosshair 

* orientation_in_tenths - orientation of rotated crosshair ir. tenths 

* of degrees 

* res - pointer to cwa_point where result will be stored 

* gross_locate_crosshair_center_in_image ( ) returns signifying 

* whether or not it found a crosshair 
*/ 

static int gross_locate_crosshair_cencer_m_image 

(cip buffer *img, int orientation_in_tenths , cva joint *res) 

{ 

cip_buffer *target=NULL; 
cse_model *mdl=NULL.; 
cct_signal sig; 
cse_results cse_res ; 
int ans ; 

NO_REGISTE?. '.target) ; 
N0_REGIS7SR (mdl. ; 

if (sig=cct_catch :oj ) 
goto done ; 

target = cip_create (GROSS JTARGZT^SZZZ . 3RCSS_7ARGE7_S IZE . 3 '; 

make_crcss_hair_center_ image ; target , orientation_m_tentns : ; 

mdl = ctr_tram_mcdel (target . Z . Z 0. 

cse_area_search img , mdl, iewa_calib_sp . atse_res 
if (cwa_f ind_target_debug_f lac i Cxi. 
printf ( "*d Vd Vd Vd\n'\ 

cse res . x , cse res . y, cse_res . score . ose_res . round; : 

resox = i. * -:GRCSS_TARGET_S;ZZ/2 - icse_res .x.n: " 

res->y = 1 . * <GRCSS_TARGE7_SrZE.' 2 * CIA_RND16 : cse_res . y . n; ■ 
done : 

if (target) cip_delete i target ) , target = NuT-L; 
if (mdl) otr_deiete_mcdel (mdl) , mdl=NUX-L; 
if (sig) cct_throw(sig) ; 

return cse_res . found; 

} 

/* compute distance from a point to the close* 

static double distance__f rom_pomt_to_line (cwa _pomt *pl. cwa_-ine 

{ 

cwa_!ine 12; 
cwajoint p2 ,- 

12.x = pl->x; 
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12. y = pl->y; 
12. t = 11->C + PI/2; 
cwa_lines_to_point (ll , &12 , &p2) ; 
return (sqrc ( (pi- >x-p2 . x) * (pi- >x-p2 . x> 
(pi- >y-p2 . y) * (pi- >y-p2 . y) ) ) ; 



/* Fit a line to an array of points */ 

static double f it_line_to_points 

(cwa_point *pts, int n_points, cwa_line *line) 

( 

int j ; 

double xsum=0 . 0 , ysum=0.0, xcent, ycent; 
double a, b, c, theta, xp, yp; 
cwa^ point *ptr; 

if (n_points < 2) 

cct_error (CWA_ERR_NO_ INTERSECT) ; 

for (j = 0, ptr - pts; ]<nj;c:.i:3; 3 — , ptr-- 
if (cwa_f ind_ targe t_debug_ flag i 0x1; 

princf ("*f Vf \n H , per- >x. p-r- >y': ; 
xsum + = ptr->x; 
ysura += ptr->y; 



xcent = xsum / n_points; 
ycent = ysum / rt_points; 

for ( j =0 , ptr = pts, a=b=c=0 . ; ; <n_cc:r.:s ; ] — , ?:r- 
xp = ptr->x - xcent; 
yp = ptr- >y - ycent; 
a + = xp * xp; 
b ^- 2 * xp * yp; 
c += yp * yp; 

} 

theta = atan2(b, a-c) / 2. On- 
line ->x = xcent; 
line->y = ycent; 
line->t = theta; 

if (cwa_f ind_carget_debug_f lag & Oxl) { 

printf ( H f it_line_tojpoints returned: \t " , 
(a*cos (theta+PI/2) *cos (theta+PI/2 } + 
b*cos (theta+PI/2) *sin< theta+PI/2) - 
c*sin(theta+PI/2) *sin (theta+PI/2) • .'njoiacs 
cd_showas( (int) line, "cwa_line" ) ; 

} 

return ( (a*cos (theta + PI/2) *cos i theta+PI/2) - 
b*cos ( theta+PI/2 i *s in (theta+PI/2) - 
c*sin ( theta+PI/2 J *sm \ theta+?~/2 ) ; /n oomcs; 
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static inc. print_list_of ^points 
(cwajjoint *val, int num_vals) 

{ 

int i; 

for (i = 0; i < num_vals ; i«--) 

priatf ("*f %f\n'\val[i; .x, vai [ij .y) ; 

} 

/* calib_orient_f rom_angle 0 recuns CALIB_VERT or CALI3_H0RIZ 

* depending upon the orientation (measured in degrees) of a line. Its 

* purpose is to determine whether we need to use a vertical or 

* horizontal quadratic subpixel edge interpolation 
*/ 

static calib_orient calibj3rient_frcm_angle {double angie_m_degrees > 

{ 

double angle_in_rad; 

angle_in_rad = angie_in_degrees * PZ ; 180 . 

if ( f abs ( cos I angie_m_rad; > 

f abs (sin ( angie_ir._rad; . 

return CA^13_VER7 ; 
returr. CALI3_HCRIZ ; 

} 

/* computes subpixel edge positions given ar. image and a neighbor-cod 

* (ap_x, ap_y ) to search for an edge. Depending upon the 

* onentaticn_m_degrees . ccrr.puze_subpixei_pcsiticn i ) will either 

* perform a vertical or horizontal subpixel edge estimation . Subpixel 

* edge positions are computed m the same manner as the boundary 

* tracker (using 7 neighboring pixels, and r differences where 

* we quadratically interpolate the edge pcsiticn from at least three 

* differences. 



* We compute a seven first differences iust to he safe. There is no 

* guarantee that the computed caliper position corresponds to the maximum 

* 1st difference (because we're using calipers with filter size 2) 



* img - cip_buffer containing image of crosshair target 

* ap_x,ap_y - application point where we sample image grey values 

* actually, we sample grey values at the application 

* point and +/- 2 from the application point (maybe we 

* should sample at */- 4, who knows 

* orientation_in_degrees - orientation of caliper which has been applied 

* to find this application point. Since we 

* only want to call subpixel interpolation 

* at the right place, we suggest using 

* on-axis calipers 

* pos_x, pos_y - pointers to doubles where 

* compute_subpixei_position { } will store result 
*/ 

int compute_subpixei_position 
(cip_buf f er *img, int ap_x, int ap_y , 

double orientation_in_degrees . double *pos_x. double *pcs_y) 
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:nt pO , pi , p2 , p3 . p4 , pS , p6 , p? ; 
mt d0,dl,d2,d3.d4,d5,d6, d_max, 
calib orient direction; 



a mir. 



if (cwa_f ind_target_debug_f lag i 0x1 ; 

pnntf { M compute_subpixel_pcsiti::r. called w;:r. to *d V;< V x 
ap_x, ap_y , orientaticn_m_degrees . pcs_:< , pcs_y • . 

if (ap_x < 4 | j 

ap_x > img- >width-4 || 
ap_y < 4 | | 

ap_y > img- >heighc -4 ) { 
return 0; 

} 



/ * should we sample horizontally cr ver: i rally 
direction = calib_orient_f rcm_angj 



: (direction == CALI3_HCR:z: { 
p0 = * ( img- >rat [ap_y-4 j -img- >x_cf f :e: - a p _ :•: .- 
p 1 = * ( img- >rat [ap_y- 3 ] ^ irr.g - >x_cf r set ~ap_x 
p2 = * ( img - >rat iap_y- 2 ] ~imc - >x_cf f set - a p _ :•: . ; 
p3 = * (img- >rat [ap_y- 1! -img- >x_of tset -ap_x . . 
p4 = * ( img- >rat [ap_y! -img- >x_cf f set -ap_x; : 
p5 = * ( img - > rat [ap_y*i] -img - >x_cf rset -ap_x. ; 
p6 = *■ ( img- > rat [ap_y-2 ] + img- >x_of f set -ap_;<; ; 
p7 = * ( img - >rat [ap_y+3i *img- >x_of f set ^ap_:<; ; 



dO=pl-pC 
dl=p2 -pi 
d2=p3 -p2 
d3=p4-p3 
d4=p5-p4 
d5=p6-p5 
d6=p7 -p6 



d_max « max ( dO , max ( dl , max l d2 . max i d3 , max i d4 , max • dS . ±S ■ ' 
d_min = min { dO , min (dl , mm ( d2 , min id3 , mir. ( d4 , min ( d5 , d6 1 ) 



( -d_min 
d0=-d0 
dl=-dl 
d2=-d2 
d3=-d3 
d4=-d4 
d5=-d5 
d6»-d6 
d max 



> d_max) { 



-d min; 



) 



*pos_x = ap_x*l. - 0.3; 
/* If we're at a maximum 



return the suboixel posi: 
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if ( d_ma_x > HUG E_THRES HOLD } ( 

if (dl = d_max i& d3 != d_max) { 

♦pos_y = ap_y ♦ ct_jparr it (dC . dl . d2 ) '65526.-:.. 
re cum 1; 

} else if (d2 = = d_max d4 : = d_max; ( 

*pos_y = ap_y + cz_parf iz :dl . dZ . d3 ) .'65536 . 
return 1 ; 

} else if (d3 == d_max &i d5 != d_max) ( 

*POS_y s a P_y + czjparf iz (d2,d3 , d4 ; .-55536 . ; 
return' 1; 

} else if (d4 d_max d6 != d_max) { 

*pos_y = ap_y + cz_parfit (d3.d4,d5) /6S536 . +1 . ; 

return 1; 
} else if (d5 == d_max) ( 

*pos_y = ap_y + cz_parf it (d4 , d5 ( d6} /6553S . r: . ; 

return 1; 

} 

/* If we're not a: a maxisur.. re- urn 3 seme;.-.:.-.: 3 wr;r.: 
*/ 

returr. 0 ; 



else 


{ 










p0 = * 


( img- 


>ra: ^ap_y 


* - img - 


>x offset 


-ap x-4 


pl = * 


(img- 


> rat [ap_y 


1 -img- 


>x_cf f set 


-ap_x - 2 


p2 = * 


( img- 


>rat iap_y 


-img- 


>x_offset 


»ap_x-Z 


p3 = * 


{ img- 


>rat [ap_y 




>x_of f set 


-ap_x- 1 


p4 = * 


{img- 


>rat (ap_y 


-img- 


>x_cf fset 


-ap_x: ; 


p5 = * 


( img- 


>rat {ap_y 


-iir.g- 


>x_of f set 


«-ap_x-l 


p6 = * 


\ img- 


>rat [ap_y 


+ img- 


>x_of f set 


-ap_x-2 


p7 = » 


( img- 


>rat (ap_y 


-img- 


>:<_of f set 


-ap_x-3 



dO=pl -pO ; 
dl =p2 -pi ; 
d2=p3-p2; 
d3=p4-p3 ; 
d4=p5-p4 ; 
d5=p6-p5; 
d6=p7-p6 ; 

d_max = max ( dC , max { dl , max { d2 , max i d3 , max i d4 , max ( d5 . d6 /)))}} ; 
d_min » min (do , mm ;dl , mm id2 , mm (d3 , min id4 , mm i dS . d6 • ) ) ) ) ) ; 

if (-d_min > d_maxi { 
dO=-dO; 
dl=-dl; 
d2=-d2 ; 
d3=-d3; 
d4=-d4 ; 
d5*-d5; 
d6=-d6 ; 

d_max = - d min; 

} 

/* If we're at a maximum, compute the subpixel position and 
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* return 1 
•/ 

*pos_y = ap_y* I . + 0.5; 

if (d_max > iDGE^THRESHOLD) { 

if (dl == d_max d3 ! = d_max> { 

*pos_x = ap_x*i. ♦ cz_parf it (dO, dl. d2,VS5 536 . -2 . ; 
return 1; 

} 

else if (d2 == d_max && d4 != d_max) { 

*pos_x = ap_x*l. + C2_parfit(dl,d2.d3 1/65536. - 1 . ; 
return 1; 

} 

else if {d3 « d_max d5 != d_max) { 

*pos_x = ap_x*l. + cz_parf it !d2 . d3 . d4 ) / 65536 . ; 
return I; 

} 

else if (d4 « d_max d6 1= d_max- { 

*pos_x = ap_x*l. * ct_parf :: 'd3 , d4 , d5 '. 6 5 5 36 . -1. ; 
re cum 1 

i 
/ 

else if • dS == d_max) { 

*pos_x = ap_x* 1 . - ct_parf it :d4 . d5 . do . /65 5 36 . -2 . ; 
return 1 ; 

) 

/ * If we're not at a rnaxirr.um. something's wrzr.z . and return D 
*/ 

return 0 ; 

/ 



\ 



enumerate_j50ints_and_f it_line_alcng_ray ( ) enumerates edge points 
along a ray from a start_pcmt tc an end_ccint by using calipers 
(and possibly subpixei interpolation; to compute edge positions 
enumerate_points_and_f it_iine_aiong_ray ( ; continues enumerating 
points until one of the following conditions occurs : 

0 . we reach the end_point 

1. no edge is found by the caliper 

2. more than one edge is found by the caliper and the highest 
scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the C0NFUSI0N_THRESHOU>) 

3. the computed edge point is an outlier (the distance between the 
edge point and the nominal line is larger than threshold) 

4. we run off the cip_buffer (screen) 

After it has enumerated the edge points, it then calls 
f it_points_to_iine to compute the optimal line estimate 

img - cip_buf fer containing a crosshair target 
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starrjDCinc, end_point - define region alcr.g whirr, zz -apply calipers 
nominal_hne, threshold - used :c decemme whether ar. edge pe- 
ls an outlier (and consequently end the 
enumeration of edge points) . The outlier 
test involves checking whether the 
distance from the pome to the 
step size - frequency of edge point sampling. Since we want tc 

evenly sample the lines, we move step_size number cf 
pels in the larger direction (and a fractional number 
of pels in the other direction) . This approach provides 
even sampling assuming we are doing on-axes caliper ir.g 
or parabolic subpixel interpolation 
search_caliper - caliper used to search for edge points 
computed_line - cwa_line where result will be returned 
enumerated_points - optional array of cwajpomts where found edge 

points would be stored 
numjoints - pointer to an int where the number of fcur.d points 
would be stored 

do_subpixei_interp - flag characterizing whether we wane cc perfc™. 

subDixel mterpclat icr. after app.yir.g cr.e calipt 



static int enumerate_pcmts_and_f i t_lme_along_ray 
(cip buffer *img, cwa_ocmt # start__pomt . cwa jzcl~z *er.t__tc mt 
cwa_lme •ncminal_line , double threshold, double step_3i=e. 
ccip caliper *search_caiiper , :va_l:r.e *ccmputed_lme , rwa_pcir.t 
♦enumerated oomts, int *num_pomt3, int dc_subpixel_ir.terp 

{ 

int i , j , Jc ; 

double maxAbsCosSm; 

cwa_pomt ap_point, *er.ux._cc mt . vec; 
int ap_x, ap_y; 

cclp__results res_h [N_£3GES * I -1 ! ; 

cclp_params epp = ( £-GE_TH?.£3HCl^:* 1 2 , N T _EIGZ5 , ? ■ . 

double ccip_angle; 

mt num_samples . tmp ; 

double caliper^ vert ; 

cct_s ignal sig ; 

cwa_l ine sampl ing_l ine ; 

if {cwa_f ind_target_debug_f lag & 0x1) { 

printf ( M enumerated_points_and_f it_lme_alcng_ray called with .n" 

printf{ M img *x start_?aint Vf \t end_point *f *f\n" , 

img , start_point - >x . s tart_pomt - >y , end _pomt - >x , end_ocmt - >y ) 

printf { n nominai_line \t Vf t \t threshold %f \n M . 

nominal_line- >x, ncmmal_lme- >y . nominal_lme- >z . threshold; ; 

} 

/* Compute the sampling line */ 
sampling_line.x = start jpomt - >x; 
sampling_line . y = startjoin: - >y ; 

sampling_line . t = atan2 (end_pcmt - >y- start _poir.t->y. 

end_poin t - >x - start _pcint - >x: ; 
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/* Compute the seep which is integral ir. x :r y rccrzir.ates 
*/ 

if ( f abs (cos (sampling_lme . ti i > fabs isin ; sampi ir.g_l ine . t 

maxAbsCosSm = fabs ices ( samp img_ line . z. ' ; 
else 

maxAbsCosSm = fabs (sin { sampling_lme . z ) ) ; 

vec.x = (cos (sampling_line . t) /maxAbsCosSin ; * szep_size; 
vec.y » (sin ( s amp ling_ line . t) /maxAbsCcsSin) * step_size; 

/* Compute the number of samples which we will be making 
•/ 

num_samples - 

max(abs( (int) ( start_point- >x * endjemt ->:<) . . 

abs((int) (start_point - >y - end_comc - >y ) 
step_size ; 

if ( ! enumerated^pcints i 
enumeratedjjcinrs = 

(cwa_pcint *) chp_ailoca {nu^._sair.ples*si = ecf cwa_:::r.: 

if ( ! num_points ; 
num_points = itmp; 

cclp_angle = sampling_lme . t * 130 . .'?:-9C . ; 

if (do_subpixel_interpi { 

if (calib orient_f rom_angle cci?_ancle: == CA^I3J-~?.T: 

cclp_angle = 0 ; 
else cclp_angle = 90; 

} 

if (cwa_f ind_target_debug_f lac & 0x1. 
print: ("ccip_angle *f\r." , cclp_angls; ; 

/* Inner Loop: Enumerate edge points 
♦/ 

for (i = 0, apjoi.it .x * start_pomt- >x, apjjcmt.y = 
starz_point - >y , enum_pomt = enumerated jicir»C3 , 
♦numjpoints « 0 ; 
i < num_samp 1 e s ; 

i++, ap^point.x += vec.x, ap_point.y *= vec.y. er.um_pcmt *- , 
(*num_points) { 

ap_x = (int) (ap_point.x * 0.5); 
ap_y = (int) (ap_point.y + 0.5); 

if (cwa_f ind_target_debug_f lag i 3x1 ; 

printf ( "ap_x Vd ap_y Vd\n" , ap_x, ac_y: ; 
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res h [0) . f eund = res_h [ 1 J. found - 0. 



/* cip_trans£orm throws CGEN_ERR_SADARG , C1?_ERR_?ELA23R . 

* CCLP_ERR_FI7 when caliper region extends past "he ccur.ds z : 

* the cip_buffer. We wane cc break off the search a: tin is 

* point anyways, so we catch the error and jump tc fittir.g a 

* line to the points 
•/ 

if (sig = cct_catch(CGEN_ERR_BADARGi ; 
goto fit_line; 

if (sig = cc t_catch(CIP_SRR_PEL*ADDR) ) 
goto fit_line; 

if (sig = cct_cacch(CCL?_ERR_FIT} ) 
goto fit_lme; 

ccip_apply 

(search caliper, imc , ap_x , ap_y . rrlr_ar.gle, *zzz res_r. 

cct_end(CCL?_SRR_Frr: ; 
cct_end(CI?_ERR_?ELACCR; ; 
cct_end:CGEN_ERR_3ADARG) ; 

/* check if edge is net found 
if ( ! res_h[0] . found) 
goto fit_line; 

/* check if mere than one edge is found 2CNFCSICJI * 
if (res_h[l; . found *& 

res Jn [0] . score < CONFUSIC^_n-G*ESKCLD*res_h ! 1 ). score ; { 
if (cwa_f md_ targe t_debuc_ flag & 0x1. { 
cd_showas ires_h, "cclp_results" ; ; 
cd_showas t ires_h i 1 ] , " cclo_resulrs " . ; 

} 

goto f it_lme ; 

} 

if (cwa_f ind_target_debug_f lag & Oxl) { 

printf i M ap_x %d ap_y %d angle tf pes Vf\n'\ 

ap_x, ap_y , cclp_angie, res_h [0] .position) ; 
if (fabs (res_h [0] .position) > l.){ 
cd_showas (res_h, u cclp_resuits'M ; 
cd_showas (&res_h[l] . "cclp^resuits" ; ; 

} 

} 

/* compute the subpixel point measured by the caliper * ' 

enum_j)Oint - >x = 

ap x + 0.5 + res_h [0] . position^cos icclp_ar.gie*?I. 130) ; 
enumjomc - >y = 
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ap_y * c.5 * res_h[c; . pes iticn*sir. rclc_ar.cl-? * ?: 

if (do_subpixei_interp) / 

if { : compute_subpixel_oosit icr. 

dmg, (int) enum_ point - >x . *'mt; er.um_po;n: - >y . 
cclp_angle, &enum_point - >x, ienum_poir.t - >y • 
goto fit_line; 
if (cwa_f ind_target_debug_f lag i Dxl) 

printf ( "compute subpixei position returned Vf tf'.n", 
enum_j)oint- >x ( enum_point- >v) ; 

} 

tfendif 

/* check if the point is within the ACCEPTABLE REGION 
* (within distance threshold of the nominal_iine 
•/ 

if (distance_f rcm_pcint_tc_lir.e •.er.uxjcir.: . ncxinal_lir.e : > 
threshold) 
goto fit_iir.e; 

fit_lme: 

/ * fit a line to the points and return : if we er.::«-*.:ered 
* any bad points, return 1 ttnerwise 

f it_lme_to_pcints 

(enumerated_pcmts , *nu:r._pcint3 . rcmcutec line: ; 

\ 

static int round_nuru;er : double x, 
{ 

if (x > 0) 

return (int) (x*0.5> 
else 

return ( mt j (x-0.5) ; 

} 

/* average two orientations but take into account that orientations 
* mod PI are equivalent 
*/ 

static double average two_or ientations * double orient 1. double orient 2; 

{ " " 

double diff.res; 
int modDiff; 

if (cwa_f ind_target_debug_f lag & 0x1) 
print f ( "avg Vf Vf M , orient_l , orient_2 ) ; 

diff = orient_2 - oriental; 

modDiff = round_number idif f .'?! ; , 

res= [ (orient 1 -orient 2 -modDif f *?I , .* 2 } ; 
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. f < cwa find target_debug_f lag 



recun res j 



*f\a w 



res j 



/ ♦ coT.puce angle in altemat 

double trans form_angle_accor; 
(priv transform *map, double 

{ 

double realSin, realCos ; 



:ve coordinates given by :r.e rtap 

:mg__to_calibracicn_map 
img_or ientaticn : 



cz_transf ormPoint (map, cos i img_cnentaticn: , sir. ( img_:r:er.:a::cr.! . 

&realCos , trealSin) ; 
return (atan2 { realSin, realCos ) ) 

} 

/* average the orientations of the four rays tc ccmpute tr.e best 

* composite orientation estimate. Giver, the orientations z : the feu: 

* image rays (the rays should be oriented m multiples of out 

* with each other.', compute the "average* cner.tat icr. . First, the 

* orientations are aligned witn img_cr lent [ Z ; sc if img_zr lent ] : ; 

* characterized a horizontal line, then all z i the angles may be 

* snifted by ?Z.'Z zz become horizontal, tnen we average tr.e result i: 

* orientations. 



* This function transforms tne image 

* orientations into the physical domain m order tc 

* constraint that the physical lines are orthogonal 
*/ 



static double average_f our_image_cr lentations 
( cva_calib *caiib, double *img_orient: 

( 



double phys_orient In ] , avg_phys_crient , avg_img_onenc , 

* First, transform angles from image coordiantes to physical 

* coordinates 
*/ 

for ( i = 0 ; i < 4; 1" ) 
phys_onent [i] = 

transf orm_angle__acccrdmg_co_calibration_map 
( { ( p r i v_c wa _c a lib calib; - >map . img_orient [i; ' ; 

if (cwa_f ind_carget_debug_f lag & 0x1 : 
for (i = 1; i < 4; i~ 

printf ( M *f \n" . phys_orient '. i". * ; 

/* Shift orthogonal angles by ?l/2 m order to line up with the 

* first orientation 
*/ 

for ii = 1; i < 4; i--:- 

if ( f abs ( s m ( phys_or lent [l] - phys_cr tent [0] ; >scrt Z.S.. 
phys_orient (i! ?I.'2; 
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/* average the orien:ac:ons * 
avg_phys_orient = 

average_cwo_orienca::ons 

( average_cwo_or:enca::cns phys_cr lent [0] , phys :r:er.: *l] 
average_cwo_or:enca:ions ;phys_onent [2 j , phys orient 12] 

/* compute the orientation in image coordinates corresponding 
* the averaged orientation 
*/ 

avg_img_orient = 

transf onn_angle_acccrding_to_caiibracion map 

( ( (priv_cwa_calib •) calib) - >inv_map, avg_phys_orient , v ; 

if (cwa_f ind_target_debug_f lag & 0x1) 

printfC'phys Vf img tf \n" , avg_phys_orient , avg_img_orient ) ; 

return avg_img_orient .- 



/* compute the image orientation fcr which the physical, cner.catic: 
* is orthogonal to another image orientation 
•/ 

static double image_ncrmal_tc_image_angl£ 
(cwa_caiib *caiib , double imc_orient 

( 



return 

( trans f crm_angle_acccrdingjtc_cal ib rat ion map 
( ( {priv_owa_calib * : calib - > ir.v_map , 
( transf orrr._angie_according_to_calibration map 
( ( (priv_cwa_calir * ' calib . - >mao . ima crient; * • 

} 

/* 

* compute_center_positicn_ar.d_angie_assuming_f cur_guaranteed_good_rays '. ) 

* improve estimate of the crosshair position and orientation by 

* searching four regions of guaranteed good length (50 pixels) 

* eminating from the crosshair center. Lines are fit to the edge 

* points along the four regions, and then the orientations of these 

* lines are "averaged" . Two orthogonal lines are computed by 

* averaging pairs of the four rays Finally, the position estimate is 

* computed by intersecting the two orthogonal lines 



* img - cip_buffer containing an image of a crosshair target 

* calib - cwa_calib object characterizing pixel non- rectangular ity 

* estimated_ctr_img - estimated position (given in image 

* coordinates) of crosshair target (computed in step II 

* estimated_orientation_phys - user- supplied orientation estimate 

* (given in physical coordinates) 

* guaranteed_good_di stance - length (in pels) of guarateec good 

* region extending outward from the origin 

* computed_center - structure where computed position will be stored 
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* computed_orier.taticn_phys - double where computed cr:er.:a:;cr. :r. 

* physical coordinates i will oe scored 

* good_rays - array of four cwa_ime structures where gocd rays 

* (lines fit to sampled edge points m the gccd region, 

* will be stored 
*/ 

static inc compuce_center jos it icn_and_angle_assuming_ four _gua rant eed_gcod_ rays 
(cip_buffer *img, cwa_calib *calib, cwa_j3aint *es timated_ttr_img . double 
estimated orientation_phys , double num_guaranteed_good_pomts , 
cwa_point *computed_center , double *computed_onentation_phys . 
cwa line *good_rays) 

{ 

int i ; 

double angle [4]; 

cwajpoint startjoint [4 ] , end_point [4 ] ; 
cwa_line computed_line [4 ] , ncminal^lme ; 
double img_angle [4 ] , angle_horiz_lme ; 

angle [0] = estimated 
angle [1] = estimated 
angle [2] = estimated 
angle [3] = estimated 

/* We define the 5C pel GUARANTEE! 3CCD REGION in terr.s zi a 

* VALID REGION so that we can use 

* the enumerate_points_and_f it_lme_along_ray • : functicn 

* If we're sampling along a line and we want tc step sampling 

* after 50 pels, then we can use a nominal line which is nc rrr.al 

* to and passes through the start point . and a threshold zz 5 C 
*/ 

nominal_line .x = estimated_ctr_img - >x ; 
nominal_lme . y = escimaced_ccr_img - >y; 

/* The passed orientation is given m physical cccrdmates * 

for (i =0; i < 4; i*-j 

angle [i] = trans f orm_angie_acccrdinc_to_calibracion_map 
( ( (priv_cwa_calib *) calib; - >inv_map, angle r .i; ) ; 

for (i = 0; i < 4 ; { 

/* The sampling region begins at the point a small distance 

* away from the crosshair center 

* (CIRCULAR - BUFFER_ZCNE_AROUND_GENTER/ to be exact, and 

* extends out to guar ant eed_good_di stance from the center 
*/ 

start_point [il .x = estimated_ctr_img- >x * 

CIRCDXAR_SUFTER_ZONE_AROUND_CENTER* cos (angle [i] • ; 
start_point [i] .y = estimaced_ctr_img- >y + 

CIRCULAR_BUFFER_2CNE_AROUND_CENTER*sin (angle [i! } ; 
end_pomt [i] . x = estimated_ccr_img- >x - 

num_guaranceed_gcoc - points*cos (angle [i • *, ; 
endjjomcti] .y = estimaced_ctr_img- >y - 



p r i en t a 1 1 cn_p hys ; 
orientaticn_phys-?:. 2 ; 
crientaticn^phys-?: . 
orientat icn^phys-PI * 1 . 5 ; 
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nutn_guaranteed_gccd_ooint3*sin (angle 1 i ] 
nommal_line. t = angle ( i] + PI/2 ; 

enumerate_points_and_f it_line_along_ray 

(img, istart joinc [i] , &end_pcmt [i ; , incnir.al_l ir.e . 
num_guaranteed_good_points , 

STEP SIZE_USED_TO_E3raMERAr£ - ?OINTS_IN*_3COD_RiC:CN»2 . . 
&clp_f ind_target_initial_search, 
icomp ut ed_l ine [ i ] , NULL , NULL , 0 ) ; 
img_angle(i] = computed_line [i ] . t ; 

cu_copy Ucomputed_line [i] , &good_rays [i] , sizeof (cwa_line; ) ; 

} 

/* average the angles */ 

angle_hori:_line = 

average_f our_image_orientations icalib. :mc_ar.cie . .- 

if t cwa_f ir.d_target_debug_f lag i 3x1; 
pnntf ( "average Vf Vf %f Vf Vf . 

img_angie [Oj , irng_angle ! 1 ! , ir.c_ar.gle [2 \ . iT.g_ar.gle [2] . 
angle_horiz_l_ne) ; 

/• average Che fir rays 

computed_line [0! .x = ( ccmpuzed_lir.e [ 0 i . x - ccmpuced_lir.e 
computed_line [01 . y = (computed_line ( 0] . y - computed_lir.e 
ccmputed_line [1] .x = (compu:ed_l:ne [1; .:< - compuced_l_ne 
ccmputed_line [1] .y = (computed_lir:e [1] .y * corr.purec_line 
ccmputsd_line [0] . t = angle_hcr_n_lir.e ; 

ccmputed_line [l! . t = image _rior-r.al_rc_iT.ace_ar.cle calic , ar.gle_r.cric_li.~e) 
'* compute the intersected pci" 

cwa lines cs jsc int ( &ccmputed_line [C! , iccTr.puted_line [I] . ccr.puted_center : ; 

/* fix the orientations of the gccd rays to correspond to the 
* "average" orientation 
*/ 

good_rays [0] . t = good_rays 12] . t = angle_horiz_lme ; 

*computed_orientation_phys = . 

transforro_angle_according_to_calibration_map 
( ( (priv_cwa_calib *)calib) - >map , angie_hori2_linei ; 
good_rays Ul . t = good_rays ( 3 ] . t = 

rransform_angle_accordinc_to_oalibraticn_i\ap 
{ ( (priv_cwa_calib *) caiib) - >ir.v_map. 
* computed_orientation_phy s t-P I / 2 ) ; 



3. -X; 2 

3; .y: '2 



Compute the two intersections between a circle and a line if the 
circle and line intersect. If the circle and line do r.ot 
intersection, signal CTA_ERR_NC_ INTER SECT ION 

/ 
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static mt mtersect_iine_ar.d_c ircie : :va_l:r.e 'line. :waj:::r.: »pt 

double radius. cva_poir.t * res_; 
cwa_pomt *res_2 ) 

( 

double angle_co_line, d:s:ance_:c_Ii.ne ( angle_:o_?c;r.:_:r._::r.e . 



ang!e_to_line = line- >t+PI/2 ; 
if ( (cos (angle_to_line) * ( 1 me- >x-pt - >x) + 
sin(angle_to_line) * ( line- >y-pt - >y> } < 0) 
angle_to_line *= PI; 
distance_to_line = distance_f rom_point_to_line (pt , line) ; 

if (distance_to_line > (radius • EPSILON_THRESHOLD) 
cct_error (CWA_ERR_NO_ INTERSECT) ; 

angle_to_point_on_line = acos (distance_tc_Ime/radius; ; 



res_l->x = pt->x + 

radius*cos (angle_to_iine+angle_tc_pcint_on_line) ; 
res l->y = pt - >y * 

radius*sir. ;angie_tc_lir.e-rar.gle_tc _pcir.r_cn_lir.e- ; 
res_2 - >x = pt->x + 

radius*ccs ;angie_to_lme-angle_ic _pc:r.:_:r._l:r.e , ; 

res_2->y = pc->y * 

radius* sir. ;angle_tc_lme -ar.gle_tc _pc ir.t_tr._l ir.e '. ; 



/* compute center_positicr._and_ar.gie_aiong_guaranteed_gccd_rays . 

* generate final estimate of crosshair position and orientation by 

* enumerating points along four rays (using four good rays as 

* thresholding lines) . We use tighter calipers because we trust the 

* positions of the good rays. 

* img - cip_buffer containing an image of a crosshair target 

* calib - cwa_calib object characterizing pixel non- rectangular! ty 

* estimated_ctr_img - estimated position (given in image 

* coordinates) of crosshair target (computed in 

* step 2} 

* good_rays - array of four cwa_lme structures where good rays 

* (lines fit to sampled edge points in the good region) 

* will be stored 

* tight_threshoid - threshold used for determining outlier points 

* with respect to the good rays 

* computed center - structure where computed position will be stored 

* computed orientation_phys - double where computed orientation (in 

* physical coordinates) will be stored 

•/ 

static int compute_center^osition_and_angle_along_guaranteed_good_rays 
(cip_buffer *img, cwa_calib *calib, cwa _pomt *estimated_ctr_img , 

cwa_Iine *good_rays , double tight_threshold. cwajom: 

♦computed center, double *ccmputed_onentaticn_phys , 

double ^feature score) 
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int i ,* 

double angle [4 ]; 

cwa_point start_pomt (4 ] , end_poir.: ( 4 ] ; 
cwa_line computed_line [4] ; 
double img_angle{4] , angle_honz_lme ; 
inc num_enumerated_pomts [4] ; 

cwa_point possible_start_points [2; . enumerat:sdj)c:n:s [4 ] [MAX_MUM_ POINTS] 
double avg_squared_error ; 

if (cwa_f ind_target_debug_f lag & 0x1) 

princf ( n compute_center_ position_and_angle_alcng_guaranteed_good_rays\n 

for { i » 0 ; i < 4 ; i-**+) { 

double distance_co_cencer , radius , orier.:at;on^away_f ram center : 

/* For each ray, we need to determine wha t the sampling 

* line should be 

* We do this by intersecting the ray with a circle 

* centered at the crosshair center 

* Since there are two intersection pc;r.::. we need zz 

* determine which cne is the real start zi the sampling 

* line 

* It turns out that we can sirr.ply pick the intersection 

* point which is closer to the tenter of the ray 
*/ 

distance_to_cencer = 

distance_f rom_point_to_lir.e • est imated_ctr_imc , igcod_rays [i* ; ; 

radius = sqrt CCIRC^IJ^_3™rER_2C>lE_AJRCt^IC_CiNTE?.* 
CIRCULAR 3UFFZR ZCNE ARCUNE C£!rTZR- 



cistance 



intersecc_lme_and_circie igocc_rays ! i ". , estinated_ctr_irr.g , 

radius , ipossibie_start_pcir.ts [01 , 
fitpossibie_start_pcints [1] ; ; 
if (cwa_point_distance 

(&possible_starc_points [0] , (cwa_point *; igcoci_rays ii] ; < 
cwa^po in t_di stance 

( &possible_start_pomts [1] , (cwa_ point *) &good_rays [i] ) ) 
cu_copy ( &passible_start_pomts [0] , istart_point [i] , 
sitecf (cwa_point; ) ; 

else 

cu_copy (&possible_start_points [1; , istart_pomt [i; , 
sizeof (cwa jpcmt :• ) ; 

/* Now, we have to determine which way we should travel along 

* the ray (good_ray [i] . t or good_ray [ i ] . t *?I 

* We determine which way by comparing the start point with 

* the center of the gccd ray 
*/ 

orientation_away_f rom_cencer = good_rays i i] . t ; 
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if ( (cos (onentation_away_f rom_cencer) * 

(good_rays [i] . x-esc;maced_c:r_img- >x. - 
sin(orientation_away_f rom_center: * 
(good_rays ti] . y-escimaced_crr_img- >y < C; 
onentation_away_f rom_center » = PI ; 

end_point[i] .x = start _point[i! .x + 

MAX_LHNGTH_DI AGONAL_ACROSS_C I ?_BUF FER * 

cos (orientation_away_f rom_center ) ; 
endjointti] .y = start_pomt [i] .y + 

MAX LENGTH_D IAGONAL — ACROSS _C I P_BUFFER ♦ 

sin (orientation_away_f rom_center) ,- 

if <cwa_find_target_debug_flag & Oxl) { 

printf ( "center Vf %f good_ray Vf Vf V: distance 
estimated_ctr_img- >x, escimacad_czr_img- >y 
good_rays[iJ . x,gocd_rays [L] . y, gccd_rays ! : 
distance_to_center) ; 

printf ; " Vf Vf Vf Vf *-n" , 

star:_pc:r.: \ . x , s:ar:_3c:r.: ; 1 1 y 
end oc:n: [ i ^ - x . er.d_pc int [ i ! . y .- 

} 



enumerace_po in t s _and_ f i t _ I me_al cng_ r ay 

(img, isnarrjpcint [i] , iend jc:n: [i] . igccd_rays [i] . 
t ight_threshold . 

STEP S I ZE_USED_TO_ENUMERATE_ALL_ POINTS . 
&clp_f ind_target_f ine_search . 
/*&clp_f ir.d_target_ccmpute_subpixel. * 
&computed_lire [ i * , fiter.umeraced_pci"C3 [ 0 ; . 
£tnum_snumerated_coir.t3 ' i] , 1; ; 
img_angle [i] = ccmouced_lme l i] 

} 

/* For each pair of opposite rays, use the closest: ?o:n:s so that 

* we use the same number of points on both sides of the 

* crosshair center 

* Since we don't know why the last edge point became invalid, we 

* skip the neighboring 2 points just to be on the safe side 
*/ 

num_enumerated_j)oints [0] = num_enumerated_points [2} = 
min (num_enumerated_points [0) , num_enumerated_points [2] 

num_enumerated_points [1] = aumjnumeratedjjc:r.:s [ 3 ] = 
min(num_enumerated_points (1] . num_enumerated_?omts [31 

/* Re -fit lines zz these points */ 

avg_squared_error = 0 ; 

for (i = 0; i < 4 ; i*- ) { 
double fit error; 



-CUTLIEP._3tJFFER ; 
■ -OUTLIER 3UFFER; 
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fit_error =» 

f it_lme_to_points 

( &enumerated_ points IL] [0] . r.um_enumeracecj:c:r.:3 \ i] . 
&computed_line [i] ) ; 
avg_squared_error + - f it_error/ 4 ; 
img_angle [i] = comDUted_line ( i] . t ; 

} 

*f eature_score = 

cz_rnap_average_squared_error_to_max_lOOO (avg_squared_error ) ; 

/* Average Che four rays by averaging the positions and 
* orientations 
*/ 



computed_line [0] .x ~ 

computed_line [0] . y = 

computed_line [1] .x = 

computed_line [1] . y = 



.me , : 



(computed_ime [0] . x + ccmputec_li.-:e [ Z ) .:<: 
( compute d_i me : Q ) . y + compute 
( compute d_ I :.ne [1] . x ♦ compute 
(computed lir.e^l] . y * compute 



. x; 



angle_horiz_iine = 

average_f cur_image_orienzaticns cal . irr.c_ar.zle 
ccmputed_line [0] . t = angle_honz_lme ; 

computed_iine [1] . t = imag e _no rrr.a l_tc_ image _ angle calib , ar.gle_hcriz_l ir.e: ; 



cwa_lines_tc_point ( _ccmputed_iine 1 0 J , &ccmputed_lme ' 1 ] . ccmputed_cer.ter - ; 

*computed_orientation_phys = 

trans f crn_angie_acccrdir.g_tc_cali_raticn_:r.at 

( ( (priv cwa_calib * ) calib? - >map , angie_horiz line : ; 

\ 

/* cwa_f ind_target ■ ) is the top level function for localizing 

* the crosshair target. It returns C if it cannot find the crosshair. 

* 1) use search to improve position estimate of crosshair center 

* 2) compute four good rays eminatmg from the origin by sampling 

* edge points within the guaranteed good region along the user 

* specified orientation 

* 3) Enumerate subpixel edge points along the four rays emulating 

* from the the origin and use the found good rays zo identify outliers 

* cwa_find_ target tries to localize a crosshair target within an 

* image and returns 1 if it was successful and 0 otherwise 

* img - cip_buffer containing crosshair target 

* calib - characterizing field of view 

* angle - initial orientation estimate of crosshair target 

* result - pointer to cwa_line structure where result will be stored 
•/ 

int cwa_f ind_target_intemal 

(cip_buffer *img, cwa_calib *calib. double angie, cva_lme 'result, 
cwa status 'status) 
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{ 

cwa_point gross _pos_estimate , improved _oos_es::ma:e. 
double improved_phys_orientation_estimate ; 
cwa_line good_rays [4 ] ; 
int i , ans ; 
cct_signal sig; 
ccm^cimer time; 
int CO. tl, t2, C3 ; 
cwa_line cmp; 

cwa_calib *calib_idencicy=NULi.; 
NO_REGISTER(calib_identity) ; 
cu_clear (status , sizeof (cwa_status) ) ; 

/* CWA_ERR_NO_ INTERS ECT I ON is used to signify when we ca~ct find 

* any edge points for an expected horizontal or vertical line 
•/ 

if ;sig = cct_catch i 0 : . 
goto dene; 

,'* expand the stack to make rccr. ftr tr.e +ZZZ tzj^L-z am/3 w-e 

* allocate 

cu_expand_stack i 16 1 ; 

if ( : result) 

result = itmp ; 

if (limg) 

cot_errcr ;CGEN_ERR_3ADARG - ; 

/* If no calibration object is passed. T.ake :ne so that there is 

* only one path through the code 

* / 

if (!calib) 

calib = calib_identity = cwa_calib_make : 0 . . 0 . , 0 . , 1 . . 1 . , 0) ; 
/* 1) use search */ 
ctm_begin ( &time) ; 

ans = gross_locate_crosshair_center_in_image 

(img, (int) (10. * angle* 190 /?: ; , igrcss _pcs_estimate ; 
if ( i ans) cct_error (CWA_ERR_NC_ INTERSECT? ; 

{ 

cip_buffer win; 
double x_edge. y_edge ; 
int edge_val , dir ; 
double angle_of f set ; 
cct_ signal sig; 
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int i ; 

double lef t [4] ={ -2 . , -2 . . 2 . , 2 . / ; 
double cop [4] ={ -2. , 2. . 2 . , -2 . } ; 
cip window ( img , &win , 

gross_pos_estimace . x- 3 5 . gross_pcs_es rirna.e . y - : 3 . "C . " : ; 

for (i = 0 ; i < 4 ; i** ) ( 

if ( cz_cebt_start_retum_success_vai 

(fiwixi. 3,3, left [i] , top[i] , -x_edge, iy_edge. iedge_val. 
idir, &angle_of fsec) ) 
break ; 

} 

seatus- >contrasc_score=cz_map_edge_val_co_max_1000 (edge_vai: . 

} 

CO = ctm_read ( &cime) ; 

if (cwa_f ind_target_debug_flag 4 Cxli { 
princf ("gross pos:") ; 

cd_showas • I int) £cgross_pcs_estimate , "cvajcir.:"' ; 

} 

/* 2) improve the ?cs::::n ar.c :r:er.:ac:cr. esci-ate ar.c rrrr.puie 
* che four good rays 

compute center _positicn_anc_ar.gle_assumir.g_f cur_?uarar.teed_gccc_rays 
(irng,~calib. &gross _cos_es::^iate, 
transform_angle_acccrd-ng_tc_ralibrat.or._map 
( ( (priv_cwa_calib •) calibi ->map. angle; . 
(double) CTAaANT£SD_GCCD_SrZE. iimproved_pcs_estimat6 . 
_improved_phys_orier.tati::r._estimace , cccd_rsys) ; 

tl = ctm_read i itimei ; 

if (cwa_f ind_target_debug_f lag i Oxi) ( 
print f ( " improved_pos_estimate : " ) ; 

cd_showas ( { int ) &Improved_pcs_estimate , "cwa_pcint M ) ; 

} 

/* 3) Enumerate edge points along those four good rays and use 
* chose lines co compute the final position and orientation estimates 

*/ 

compuce_cencer_position_a^d_angie_along_guaranteed_good_rays 
(img, calib, &improved_pos_estimace , gcod_rays , TIGHT_*.*IRESHC13 , 
(cwa_point *) result . £ ( result - >t J , 
StStacus- >f eacure_score) ; 

t2 = ctm_read ( itime) ; 

cz_transf ormPoint 

v ( (priv_cwa_calib v» calib } ->map, 
result- >x, result- >y , 
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iresult- >x, ^result- >y) ; 

if (cwa_f ind_target_debug_f lag & Oxii ( 
print f ("result: ") ; 

cd_showas ( (int) result, "cwajcmc" 1 ; 

} 

if (cwa_f ind_target_debug_f lag i 0x4) ( 
printf ("Times:") ; 

printf( M gross locate: %d\n M ,tO); 
printf(" fine locate: Vd\n" , ti- tO) ; 
printfC* finer locate: Vd\n" , z2- tl) ; 

} 

done : 

cu_expand_stack ( - 16 ) ; 

if (calib_identity) cwa_calib_deiete ( calib_identi:y) ; 
/* Make returned angle agree with given angle argument 

if (sig) cct_throw (sig) ; 

for (i - -8; i <= 8; i ~~ ; 
if ( (i != 0) 

( f abs (result - >t * i + 9Z.Z - angle: < f abs . result - > 
result - >t «■ = i*PI. 2 , i - - .- 

return 1; 

} 

int cva_f ind_targe t 

(cip_buffer *img, cwa_calib *calib. double ancle. cwa_lir.e 
cwa_scatus * status) 

{ 

cwa_status tmp_status; 
cct_signal sig; 

if (1 status) status = &tmp_status ; 

cct_catch(CGEN_ERH_BADARG) ; 
cct_catch (CWA_ER^_BACKI.IT) ; 
cct_catch(CWA_ERR_TRACKER) ; 

if (sig=cct_catch(0) ) { 

if (sig == CGEN_ERR_BADARG ) ( 

status- >error_f lags i = CWA_ERR_3ADARG ; 
cct_end(sig) ; 
cct_error (sig) ; 

} 

status- >error_f lags | = CWA_ERR_F AILZD_TC_FIND_7ARGET ; 
return status- > found = 0; 

} 

cwa_f ind_target_intemal ( img, calib, angle , result, status ) 
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Described herein are calibration targets and methods for determining the location 
and orientation thereof meeting the objects set forth above. It will be appreciated that the 
embodiments shown in the drawings and discussed above are merely illustrative of the 
invention, and that other embodiments incorporating changes therein fall within the scope 
of the invention, of which we claim: 
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1 . A machine vision method of determining a location of a reference point in an 
image of a calibration target, the method comprising the steps of 

A. generating an image of a target that includes 

i. two or more regions, each region being defined by at least two linear 

edges that are directed toward a reference point; 

iL at least one of the regions having a different imageable characteristic 
from an adjacent region; 

B. identifying edges in the image corresponding to lines in the calibration target; 

C. determining a location where lines fitted to those edges intersect. 

2. A method according to step 1, wherein step (B) comprises the step of identifying 
edges in the image by applying an edge detection vision tool to the image. 

3. A method according to step 1, wherein step (B) comprises identifying edges in the 
image by applying a caliper vision tool to the image. 

4. A method according to step 1, wherein step (B) comprises identifying edges in the 
image by applying a first difference operator vision tool to the image, and by 
applying a peak detector vision tool the an output of the first difference operator 
vision tool 

5. A method according to step 1, wherein step (B) comprises identifying edges in the 
image by applying a caliper tool to the image beginning at an approximate location 
of the reference point. 
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6. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a Hough line vision tool to the image 
and finding an intersection of lines identified thereby. 



5 7. 



10 8. 



A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a correlation vision tool to the image 
using a template substantially approximating an expected pattern of edges in the 
image. 

A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a projection vision tool to the image 
along one or more axes with which the edges are expected to be aligned. 



9. A method according to claim 1, comprising the steps of 

15 

identifying edges in the image corresponding to lines in the calibration target; and 



determining an orientation of the target based on the angle of those identified 
edges. 

20 

10. A machine vision according to claim 1, wherein 

step (B) includes the step of applying a Hough line vision tool to identify edges in 
the image corresponding to lines in the calibration target; and 

25 

the method further including the step of determining an orientation of the target 
based on the angle of those identified edges. 

11. A machine vision according to claim 1, comprising 

30 

A. applying a Sobel edge tool to the image to generate at least a Sobel angle image; 
and 
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B. taking an angle histogram of the Sobel angle image to determine an orientation of 
the target. 

12. A machine vision method of an orientation of a calibration target in an image, the 
method comprising the steps of 

A. generating an image of a target that includes 

i. two or more regions, each region being defined by at least two linear 

edges that are directed toward a position; 

iL at least one of the regions having a different imageable characteristic 
from an adjacent region; 

B . identifying edges in the image corresponding to lines in the calibration target; 

C. determining an orientation of the target based on the angle of those identified 
edges. 

13. A method according to claim 12, wherein 

step (B) includes applying a Hough line vision tool to the image to identify therein 
edges in the image corresponding to lines in the calibration target; and 

step (C) includes determining an orientation of the target based on the angle of 
those identified edges. 

14. A method according to claim 12, wherein 

step (B) includes applying a Sobel edge tool to generate at least a Sobel angle 
image; and 
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step (C) includes taking an angle histogram of the Sobel angle image to determine 
an orientation of the target. 



15. A method according to any of claims 1 and 12, wherein step (A) includes the step 
of generating an image of a target in which each region has any of a different 
color, contrast, brightness and stippling from regions adjacent thereto. 

16. A method according to claim 15, for use with a machine vision system having 
image capture means for capturing an image of the target, wherein each region has 
a different characteristic in such an image than the regions adjacent thereto, 

17. A method according to any of claims 1 and 12, wherein step (A) includes the step 
of generating an image of a target for which the reference point is at a center 
thereof. 



18. A method according to claim 17, wherein step (A) includes the step of generating 
an image of a target for which the linear edges substantially meet at the reference 
point. 

19. A method according to any of claim 1 and 12, wherein step (A) includes the step of 
generating in image of a target having four imageable regions. 

20. A method according to claim 19, wherein step (A) includes the step of generating 
an image of a four-region target in which the at least two linear edges of each of 
the regions are perpendicular to one another. 

21. A machine vision method of determining a location and orientation of a calibration 
target in an image thereof, the method comprising the steps of 

A. generating an image of a target that includes 
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i. two or more regions, each region being defined by at least two linear 

edges that are directed toward a reference point; 

n.. at least one of the regions having a different imageable characteristic 
from an adjacent region; 

B. generating, based on analysis of the image, an estimate of an orientation of the 
target therein; 

C. generating, based on analysis of the image, an estimate of a location of the 
reference point therein; 

D. refining, based on analysis of the image, estimates of at least one of the location of 
the reference point in the image and the orientation of the target in the image. 

22. A method according to claim 21, wherein step (B) includes one or more of the 
following steps: 

i. applying a Hough line vision tool to the image to find an angle of edges 
in the image; 

ii. applying a Sobel edge tool to the image to generate Sobel angle image, 
generating a histogram of the angles represented in that Sobel angle 
image, and determining a one dimensional correlation thereof; and 

iii. inputting an orientation from the user. 

23. A method according to claim 22, wherein step (C) includes one or more of the 
following steps: 
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i. applying a Hough vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 

5 ii. applying a projection tool vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 

iii. applying a correlation vision tool to the image to find approximate 
10 coordinates of the reference point; and 

iv. determining a sum of absolute differences to find approximate 
coordinates of the reference point. 



15 24. 



25 25. 



A method according to claim 23, wherein step (D) includes the step of invoking 
the steps in sequence, at least one time: 



i. applying a caliper vision tool along each of the expected edges in the 

image, where those edges are defined by a current estimate of the 

20 reference point location and a current estimate of the target orientation; 

and 

ii. fitting lines to edge points determined by the caliper vision tool. 



30 



A method according to claim 24, wherein step (D)(i) includes the step of applying 
the caliper vision tool along each of the expected edges until any of the following 
conditions is met: no edge is found by the caliper vision tool at a sample point; 
more than one edge is found by the caliper vision tool at a sample point; a distance 
between a sample edge point and a nominal line is larger than a threshold; or, the 
caliper vision tool window for the sample edge point extends outside of the image. 
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